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1. INTRODUCTION 


1.1 Background 

Monitoring the engines on today's commercial transport aircraft (either manually or 
electronically) is an extremely complex task. Even though the technology exists to pro- 
vide detailed engine information, it is not often incorporated into today's transport air- 
craft, at least in a form that the crew can readily use. Because most engine parameters 
are graphically displayed near the pilot's primary field of view, they have not been in- 
cluded in the integrated crew alerting system. Furthermore, to evoke an alert in current 
cockpits, an abnormal event (e.g. engine fire) must occur or an instrument/sensor 
threshold must be exceeded. It is possible that many abnormal engine situations could 
be detected early or avoided altogether if the appropriate engine parameter information 
could be provided to the crew, correlated wi th in formation about other parameters. 
Finally, the presentation of most engine parameters does not vary based on the airplane 
state or take into account prior values. For example, although the red-line limit for en- 
gine temperature during engine start is different from the red-line limit for cruise op- 
erations, the red-line limit displayed for this parameter does not vary. In addition, the 
length of time that engine temperature or other parameter limits can be exceeded de- 
pends upon how much they are exceeded. Exceedance values differ for each engine and 
aircraft type, and are difficult to memorize and monitor. All these factors are made 
more difficult by the lack of a reference to help the pilot to determine what the expected 
range of values of a parameter is for specific conditions. 

The system hardware and interfaces required to permit the crew to monitor the perfor- 
mance of their engines have been part of a continuously evolving technology. One of the 
first examples was the simple cycle counter that stored the number of engine start-ups 
and shut-downs. Today, a microprocessor-based "black box" records multiple engine 
parameters and warns of exceedances. Hardware and software are currently being de- 
veloped (primarily for maintenance) that could develop into a third generation engine- 
monitoring system for the flight deck. One goal of this effort is to use engine modeling 
techniques which permit current parameter readings to be compared to expected, or 
"nominal," values. In developing the crew interfaces for such systems under a "human 
centered automation" philosophy, consideration must be given not only to the crew infor- 
mation requirements but also to the operational environment in which the system will 
be used. The amount and type of information presented to the crew should be situation 
dependent. Too much information, or information that is over- or under-processed, can 



cause degradation in crew decision making and problem solving as easily as too little in- 
formation. At least three levels of information analysis can be identified for considera- 
tion: 1) system monitoring, which permits trend identification, anticipation of required 
actions, and planning; 2) alerting information, which gets the attention of the crew and 
provides identification of an out-of-tolerance condition; and 3) guidance information, 
which advises the crew of the actions appropriate for the situation. The focus of the cur- 
rent study is the display of information that can facilitate the monitoring and problem 
identification functions. 


Parameter 

values 



Upper limit (red) 

Upper warning (red) 
Upper caution (yellow) 
Normal (green) 

Lower caution (yellow) 

Lower warning (red) 
Lower limit (red) 


© This column (green) shows a slightly low 
deviation. 


© This column (yellow) shows a deviation 
into the caution 
region. 


Figure 1 . Example of Engine-Monitoring Display Element Derived 
from Task-oriented Design Process (from Abbott, 1989) 


NASA has developed a "Task-Oriented Display Design” process (Abbott, 1989) which 
modifies the traditional design process by identifying and providing only relevant infor- 
mation in a form that is appropriate to the user’s task. The application first chosen to 
demonstrate and evaluate this process was aircraft engine instruments because such 
instruments provide both a control task and a systems monitoring task. The result of 
the initial effort was an integrated engine parameter display that used bars that "grew” 
upward or downward from a horizontal line that represented "nominal" values to show 
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engine parameter levels, and changed colors to show exceedances (Figure 1). One of the 
conclusions of this effort to compare traditional and task-oriented-designed displays was 
that "the overall results... showed a favorable i nc rease of both the user's subjective as- 
sessment and failure detection rate (and therefore a reduction in what is typically 
termed 'operator error') for the task-oriented-design display." The results also con- 
firmed the premise that "providing information that is tailored to the user’s task, both in 
content and form, increases the user's ability to utilize that information" (Abbott, 1989). 
The goal of the current study was to modify the application of the NASA task-oriented 
design concept to incorporate it into currently operational "glass cockpit" engine dis- 
plays, and to evaluate it using pilots in a full mission simulation. 

1.2 Candidate Display Concepts 

Five candidate engine-monitoring display formats were developed for this study using 
the Engine Indicating and Crew Alerting System (EICAS) displays currently used in 
Boeing advanced cockpits (757, 767, and 747-400) as a baseline. The five formats were 
designed to evaluate and compare the operational utility of adding three types of task- 
oriented information to the basic EICAS displays: engine parameter alert (caution and 
warning) messages, messages to "monitor" an engine parameter that deviates from 
expected values, and a graphic depiction of the range of nominal or expected values for 
the parameter for current conditions. The five display formats represent the major 
factor in the experimental design: Display Condition. Briefly, the five display formats 
are as follows: 

Display Condition 1: Basic EICAS. Conventional EICAS engine displays; color 
changes occur when a parameter exceeds caution and/or warning limit(s); alpha- 
numeric caution and warning messages appear for systems problems, but not for 
the graphically-displayed engine parameters. 

Display Condition 2: Basic EICAS with engine parameter alert messages. Similar to 
Display Condition 1, except that alphanumeric alert messages are added if the 
value of any graphically-displayed engine parameters exceed caution and warning 
limits. 

Display Condition 3: Augmented EICAS with green bands. Similar to Display Con- 
dition 1, except that an engine model augments the information available about 
engine performance and provides "nominal range" data for each engine parame- 
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ter. The nominal range is shown by adding "green bands" to the graphical display. 
Color changes occur when the value of any parameter is outside the nominal 
range; no alphanumeric messages appear for the depicted engine parameters. 

Display Condition 4: Augmented EICAS with engine parameter alert and "monitor 
parameter" messages. Similar to 3, except that the nominal ranges generated by 
the model are not depicted graphically. "Monitor parameter" messages and color 
changes occur when the value of engine parameters are outside the nominal 
range; engine parameter alert messages appear when parameter values exceed 
caution and warning limits. 

Display Condition 5: Augmented EICAS with green bands, and engine parameter 
alert and "monitor parameter” messages. Combines features of Display Condi- 
tions 3 and 4; includes green "nominal range" bands, and both engine parameter 
alert and "monitor parameter" messages when appropriate. 

The display conditions are described in more detail in section 5.2, and examples are 
shown in Figures 4 through 10. It should be noted that the display formats as imple- 
mented in this study are not necessarily sanctioned by Boeing. 

2. STUDY OBJECTIVES 

The overall objective of this effort was to develop operationally-viable display concepts for 
an engine monitoring system that would enable the crew not only to monitor engine 
trends more effectively but also to detect trends in multiple parameters. The specific ob- 
jective for the current study was to evaluate the five EICAS display variants in a simu- 
lated operational environment under various non- normal conditions. In doing so, the 
goals were to: 

a) Evaluate the concepts for performance differences. 

b) Evaluate which, if any, of the features added to the basic EICAS format were useful or 
desirable. 

c) Evaluate the perceived workload associated with each concept. 

d) Determine what changes are recommended by the user community for engine moni- 
toring displays and their implementation. 
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3. SUBJECTS 


A total of 10 experienced transport pilots participated as subjects in the concept evalua- 
tion test. Because the test was to be conducted in a medium-fidelity research simulator 
equipped with a Pratt and Whitney 2037 two-engine model and EICAS, subject selection 
was restricted to pilots who were type-rated in the 757, which incorporates the EICAS 
displays and uses the Pratt and Whitney 2037 engine. It was hoped that this restriction 
would permit the subjects to make more meaningful judgements about the operational 
acceptability and perceived workload of the system. Three of the pilots who acted as 
subjects in the study were current line pilots with a major U.S. airline; the other seven 
were Boeing Flight Training pilots who could provide input from a training perspective. 
Data about the subjects and their experience is summarized in Table 1. 

4. FACILITIES 

A proper evaluation of any aircraft display or control system involves the problem of 
realistically duplicating the operational conditions (environment, workload, procedures, 
perception, etc.) under which the display or system is generally used. The amount of 
realism required to accomplish the evaluation is dependent on the objectives of the test. 
The environment created by the facility must be realistic enough to generate data which 
satisfy these objectives. In the current study, the technologies being evaluated required a 
facility capable of supporting advanced display concepts and providing a realistic envir- 
onment for their operation. The following sections describe the test facility and the com- 
ponents that were used in the study 

4.1 Engineering Flight Simulation Center 

The Engineering Flight Simulation Center (EFSC) is a portion of the Flight Systems Lab- 
oratory which is the principal flight simulation and avionics test facility used in the de- 
sign, development and certification of Boeing Commercial Air Transports. It is one of 
the foremost engineering simulation facilities available in the industry and can support 
all phases of avionics system development and testing. The laboratory contains all the 
components necessary to conduct a complete man-in-the-loop real-time aircraft simu- 
lation. It includes the Flight Deck Research Cab which provides a flexible tool for re- 
search and development efforts. 


5 


irniniu 

IIBUffllH 


h- h- o o 
in o o o 

CM i- 


TJ- CD CO CD O) 

in co o co 


tt co co ^ 


CNI o O O CO 
CM 


o> |=> 


00 I CD 


CD CD 


00 CM P P P 


O CO Tf o LO 

™ c\i 


o o o o 

CM 


O O O CO 


m I CD 


<0 |3 


CD O 

CM CM ° ° 


CM 1 3 


O I LO I CD I O I CO 


o o o o in 

CM 


I- ICQ 

O I » 

KiJ I CD 

"D 

CD 
D 
CO 


^ c a ^ 

c/) (0 CO ^ 

■*“» "q. ■4-' W 

C cB C C 
® o <D T 
CO V « 

CD O 


IS’®! 

I j= i ® © 
w 2 2 § £ 
» 1 5 (JO 
to 1 1 

© 

>- 


i *, S ® a. - S 

c m « ^ C *§ « 

f .1 I°il | 

«<«©«<« 

• E * 

3 


^ c ■§ 

cm® 

8 ® c 

« 5 CO 


Table 1. Summary of Subjects Experience 

















































































































































































































4.2 Flight Deck Research Laboratory and Cab 


The Flight Deck Research Laboratory is a basic engineering laboratory that provides re- 
search and development capabilities to facilitate the progressive evolution of new display 
and control concepts. It was established to provide for systematic increases in simula- 
tion and technological capabilities; and to provide part-task demonstration and evalua- 
tion. This laboratory provides capabilities to support (1) early laboratory work requiring 
use of bench development and test facilities, (2) successive stages of partial simulation 
using simplified approximations of sensor and aircraft systems, and (3) concept imple- 
mentation for full simulation to confirm the application. 

Associated with this laboratory is the flexible all-electronic Research Cab (shown in Fig- 
ure 2), which has been developed to fulfill a dual purpose. First, the cab provides a facil- 
ity to appraise the requirements for an individual display or control, including both con- 
text (formats) and content (information requirements). It also permits the preliminary 
evaluation of dynamic display formats to ensure that the pilot receives the information 
quickly and accurately. Secondly, the cab provides the facility to initiate systems integra- 
tion which is necessary in the development of new displays and controls. The cab facili- 
tates evaluation of the degree to which new concepts meet flight deck system require- 
ments. The cab also provides the facility to conduct architectural integration of new dis- 
play/control concepts. As a system integration facility, the cab has become a concept de- 
monstrator and the foundation for the development of advanced flight deck applications. 

The cab instrumentation used in the current study is shown in Figure 2. The configura- 
tion included D-size display units (6.5" by 6.5" display area) for both the Electronic Flight 
Instrument System (EFIS) and the Engine Indicating and Crew Alerting System 
(EICAS) displays. These displays were driven by the Evans and Sutherland advanced 
Display Development System (DDS) which permits rapid prototyping of display formats, 
and provides flight- quality graphic formats that can be easily reconfigured for research 
and development testing. The flight control system used in the cab for this study was a 
wheel/column implementation with hydraulic "feel" system. This cab was provided 
with a switchable day/night Computer Generated Imagery (CGI) visual system for out- 
the-window displays. This is a two viewpoint CT-5 system manufactured by Rediffu- 
sion/Evans and Sutherland, and provided the pilot and co-pilot/observer with a front- 
window-collimated display. Each of these displays comprised a color TV monitor, beam 
splitter, and spherical mirror. 
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Figure 2. Research Cab and Instrumentation 


5. METHODOLOGY 


5.1 Design and Rationale 

The test described in this section was designed to evaluate the effectiveness and viability 
of using engine models as a means of providing information to enable the pilot to moni- 
tor engine parameters more quickly and accurately in the operational environment, and 
to evaluate different formats for providing the information. The basic experimental de- 
sign for the study is presented in Figure 3. The test compared the five EICAS display 
formats on the basis of both objective data (subject response time and accuracy) and sub- 
jective data (subject ratings). The test design was a 5 x 2 x 2 factorial within-subject de- 
sign with repeated measures on all three independent variables. The "Display Condi- 
tion" variable had five levels; the "Number of Engines" variable had two levels; and the 
"Problem Onset Time" variable had two levels. 

5.2 Display Conditions 


The Display Condition variable had five levels. The first level was the standard EICAS 
(with slight modifications) used on Boeing 757 aircraft equipped with Pratt and Whitney 
2037 engines. Each of the other display conditions had one or more features added to the 
basic EICAS display format. Table 2 summarizes the characteristics of each display 
condition. 

5.2.1 Display Condition 1: Basic EICAS. 

Features described in this section were common to all display conditions; each of the 
other conditions added one or more features to the Basic EICAS display format. In the 
basic (standard) EICAS display used in this study, information about engine parameters 
is shown in either round-dial or vertical-scale formats on two screens in the center front 
of the cockpit. Figure 4 shows the upper and lower EICAS screens when no engine pro- 
blem exists. (All other figures in this section will show the display features present 
when an engine problem occurs, and include only the upper screen if appropriate.) In 
the case of round-dial indicators (EGT, for example), a pointer shows the current level of 
the parameter along the circumference of the dial. Red and amber tick marks along the 
circumference mark the upper and (if appropriate) lower caution and warning limits 
for the parameter. For vertical-scale parameters (oil temperature, for example), a caret- 
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Display Conditions 


1 . Basic EICAS 


2. Basic EICAS 
with Alert 
Messages 


3. Augmented EICAS 
with Green Bands 


4. Augmented EICAS 
with Green Bands, 
and Alert and 
"Monitor" Messages 


5. Augmented EICAS 
with Green Bands 
and Alert and 
“Monitor" Messages 



Number of Engines with Fault(s) 


Subjects 


Figure 3. Basic Experimental Design 
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type pointer shows the current level of the parameter along the length of a vertical line; 
upper and lower caution and warning limits are shown, as appropriate, as tick marks 
at the top or bottom of the vertical scale. 


DISPLAY CONDITION 

1 

2 

3 

4 

5 

FEATURES 

Basic EICAS 

Basic EICAS 
with Alert 
Messages 

Augmented 
EICAS with 
Green Bands 

Augmented 
EICAS with 
Alert and 
Monitor 
parameter 
Messages 

Augmented 
EICAS with 
Green Bands 
and Alert and 
Monitor 
parameter 
Messages 

Alphanumeric alerting 
messages for non-engine 
systems problems 

X 

X 

X 

X 

X 

Alphanumeric alerting 
messages (caution/ 
warning) for engine 
parameters 


X 


X 

X 

Alphanumeric "monitor 
parameter" messages for 
engine parameters 




X 

X 

Green bands to depict 
"nominal range" for each 
engine parameter 



X 


X 

Pointer and digital 
readout/box change 
color only when engine 
parameter exceeds 
caution/warning limits 

X 

X 




Pointer and digital 
readout/box change 
color when engine 
parameter value departs 
from “nominal range" 



X 

X 

X 


Table 2. Display Condition Features 


11 






















Figure 4. Example of Display Condition 1 : 
Basic EICAS, All Parameters Normal 
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For both round-dial and vertical-scale formats, these limits are normally "hard" limits; 
that is, they are set for the particular type of engine, and do not vary based on the current 
conditions or the operational history of the engine. A box with a digital readout also 
shows the current value of the parameter for each of the graphically-displayed engine 
parameters. 

When the current value of the parameter exceeds caution (amber) or warning (red) lim- 
its, the pointer and the digital-readout box and number turn amber or red, respectively 
(see Figure 5). Information about non-engine systems status is presented in the form of 
color-coded alphanumeric caution, warning, and status messages in the top left comer 
of the upper display. These messages are listed in order of priority (first warning, then 
caution and status messages) and accumulate if necessary. No alphanumeric mess- 
ages are presented in the basic EICAS system for the graphically-displayed engine- 
parameters. 



Figure 5. Example of Display Condition 1 : 
Basic EICAS, Left-Engine High EGT 
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The basic EICAS display format used in this study (Display Condition 1) differed from 
the standard EICAS presentation used on Boeing aircraft equipped with Pratt and Whit- 
ney 2037 engines only in the following ways. First, under most conditions, engine infor- 
mation is presented in a compact form only on the upper (of two) EICAS display screens 
in the aircraft, and the lower screen is blank. If a problem occurs that involves informa- 
tion normally presented in its full format on the lower screen, the lower screen informa- 
tion is automatically presented. During this study, however, EICAS information was 
always shown in full format on both screens. Second, in the standard EICAS format 
used on Boeing aircraft, the entire round-dial and vertical-tape scales that represent 
engine parameters are white. In this study, the portion of these scale that represented 
caution and warning conditions (above the upper, or below the lower, caution and warn- 
ing limits) for engine parameters were colored amber and red, respectively. Finally, in 
the standard EICAS system, aural tones accompany the presentation of alphanumeric 
caution and warning alert messages. In the present study, aural alerts occurred only 
under the limited conditions described in section 5.3.2. 

In this study, the EICAS display format for a Boeing 757 equipped with a Pratt and Whit- 
ney 2037 engine was used as the baseline. It is representative of current airline systems 
and was used so that realistic operational data could be obtained for comparison. As 
noted previously, this display was used in the expanded mode (two screens) so that all 
the parameters would be presented at all times. For the Pratt and Whitney 2037 engine, 
simulated in this study, the following parameters were presented. On the upper screen, 
Engine Pressure Ratio (EPR), Nl, and Exhaust Gas Temperature (EGT) were shown in 
a round-dial format. On the lower screen, N2 and Fuel Flow (FF) were shown in a 
round-dial format, and the following parameters were shown in a vertical-scale format: 
oil temperature, oil pressure, oil quantity and engine vibration. For all parameters, the 
maximum/minimum caution and warning limits were marked on the scales with am- 
ber or red tick marks. When a specific parameter caution or warning limit was exceed- 
ed, the pointer (or caret), and the numeric readout and box changed to amber or red, as 
appropriate. No other alert indication was presented with the color change. 

5.2.2 Display Condition 2: Basic EICAS with engine parameter alert messages. 

The second display condition for this study integrated engine parameter information 
into the alert message system by adding alphanumeric messages about engine para- 
meter exceedances. When a parameter limit (upper or lower) was exceeded, an appro- 
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priate caution or warning alert message (e.g. L ENG Nl) appeared in the top left comer 
of the upper EICAS display, where caution, warning and status messages appear for 
other system alerts. This alert message was in addition to the standard change in color 
from white to amber or red for the pointer and the box with digital readout. An example 
of Display Condition 2 is shown in Figure 6. In this example, Nl is in the caution zone 
for the left engine. 



Figure 6. Example of Display Condition 2: 
Basic EICAS with Engine Parameter Alert 
Messages, Left-Engine High Nl 


One hypothesis of the study was that under high workload conditions, the addition of an 
alphanumeric engine parameter caution or alert message would benefit the detection of, 
and identification of, engine faults. This could result partially from the salient location 
of the message, which is in the top left comer of the upper EICAS display, closest to the 
primary flight displays. Additionally, placement of the message in the top left corner 
would make the display conditions associated with engine problems more similar to that 
of problems with other systems. If subjects are accustomed to seeing most caution- and 
warning-level problems identified by alert messages displayed in a consistent color- and 


15 




spatially-coded alphanumeric form in a specific location, it may take longer to detect and 
respond to problems not identified in that manner. 

5.2.3 Display Condition 3: Augmented EICAS with green "nominal range" bands 

In the third display condition, a graphic display of the range of acceptable nominal en- 
gine values for the current conditions for each parameter was added to the basic EICAS 
display to allow the user to compare the current value of the parameter with the nomi- 
nal, or expected, range. This range was generated by an engine model running as a 
background routine of the simulation software. This engine model was a simplified 
Pratt and Whitney 2037 engine, developed by Boeing with data provided by Pratt and 
Whitney. The model incorporated effects of airspeed, altitude, air temperature, air 
pressure, and throttle position. It was used to predict a range of values of each parame- 
ter that was expected ("nominal") for a given flight phase or operating condition. A 
more sophisticated model could also take into account, for example, the operational his- 
tory of a given engine. Comparing current parameter values to those generated by the 
model permitted the system to detect not only when parameter caution and warning 
limits were exceeded, but also when parameter values deviated significantly from nom- 
inal range values. A dynamic "green band" along the inside of the round-dial format, or 
along the sides of the vertical -scale format, showed the extent of the nominal range. In 
this study, the dynamics of the green band were simplified. The bands moved as appro- 
priate around the circumference of the round-dial indicators, or up and down the side of 
the vertical-scale indicators. However, the green bands for a particular parameter did 
not vary in their extent (arc or vertical length) as would be expected in a fully dynamic 
model. 

In this display condition, the location of the current value of the parameter relative to the 
nominal range could be determined by comparing the location of the pointer (for round- 
dial indicators) or the caret (for vertical-scale indicators) to that of the green band. It 
was assumed that the graphic display of the nominal range would make it easier and 
quicker to determine if a parameter was starting to deviate from normal, and whether 
that deviation was high or low for the current conditions. The pointer or caret, and the 
numeric readout and box, changed color to amber whenever the value of the parameter 
went outside the nominal range; when this happened, the position of the pointer was al- 
so outside area indicated by the green band. The pointer, box, and readout also changed 
to amber or red, as appropriate, if a caution or warning limit was exceeded (if it had not 
already done so). In this display condition, no alphanumeric alert message appeared if 
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an engine-parameter limit was exceeded. By comparing performance using Display 
Condition 3 to the performance using either Display Condition 1 (basic EICAS) or Dis- 
play Condition 2 (basic EICAS with engine parameter alert messages), the hypothesis 
that the green "nominal range" bands aid in detecting or identifying the direction of 
engine parameter deviations can be tested. Figure 7 shows a representation of Display 
Condition 3. In this example, left-engine N1 is outside the nominal range, but not yet in 
the caution range. 



Figure 7. Example of Display Condition 3: 
Augmented EICAS with Green Bands, 
Left-Engine N1 Outside Nominal Range 


5.2.4 Display Condition 4: Augmented EICAS with engine parameter alert and "monitor 
parameter" messages. 

As in Display Condition 3, an engine model was incorporated into Display Condition 4, 
and a process compared the values generated by the model with current values. In this 
display condition, however, the range of the nominal values was not displayed graphi- 
cally. Instead, if the value of one of the engine parameters went outside the nominal 
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range, an appropriate "monitor parameter" message was generated, e.g. "MONITOR L 
ENG Nl," and the pointer and the box with digital readout changed from white to amber. 
If the value of the parameter continued to deviate and exceeded caution or warning lim- 
its, an appropriate engine-parameter alert message appeared in addition to the "moni- 
tor parameter" message. This display condition, without the green bands to graphically 
indicate the range of nominal values, might not permit subjects to identify easily the di- 
rection of the fault (i.e. high or low values) but it could draw attention to the problem 
parameter and engine. An example of Display Condition 4 is shown in Figure 8. In this 
example, Nl is outside the nominal range (not depicted), but not yet in the caution range. 



Figure 8. Example of Display Condition 4: 
Augmented EICAS with Alert and "Monitor Parameter" 
Messages, Left-Engine Nl Outside Nominal Range 


5.2.5 Display Condition 5: Augmented EICAS with green bands, and engine parameter 
alert and "monitor parameter" messages. 

The final display condition combined features from Display Conditions 3 and 4. The dis- 
play of nominal ranges was the same as in Display Condition 3: nominal ranges were 
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shown as green bands along the circumference of the round-dial parameters, and along 
the sides of the vertical-scale parameters. The form and timing for alphanumeric 
"monitor engine parameter" and caution and warning alert engine parameter mess- 
ages were identical to that in Display Condition 4. Figure 9 shows a photo of Display 
Condition 5 in the Research Cab with right-engine N1 in the caution range. Figure 10 
shows a line-drawing example of Display Condition 5 with right engine oil pressure (on 
the lower screen) outside the normal range but not yet in the caution range. 



Figure 9. Example of Display Condition 5 in Research Cab: 
Augmented EICAS with Green Bands, Engine Parameter Alert 
and "Monitor Parameter" Messages, Right-Engine N1 in Caution Range 
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- - - = color change 


Figure 10. Example of Display Condition 5: 
Augmented EICAS with Green Bands, Alert and 
"Monitor Parameter" Messages, Right-Engine Oil 
Pressure Outside Nominal Range 
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Research has shown that a significant portion of response time variance is generated by 
the time it takes to detect the problem (Boucek et al., 1980). In normal operation, an aur- 
al warning accompanies the presentation of any alphanumeric caution or warning 
message on the basic EICAS display and aids in detecting the problem. However, a 
decision was made to eliminate the aural warning in this study for all the caution and 
warning alert messages (both engine and non-engine) in all display conditions. Aural 
warnings were eliminated because they could act as a consistent cue that a non-normal 
event had occurred and invalidate the use of the response-time dependent variable. 

5.3 Number of Engines with Faults 

The second variable that was manipulated in the study was the number of engines that 
exhibited an engine parameter fault. Half the engine-related problems that occurred 
affected only one engine, and half affected two engines. Whenever a two-engine problem 
occurred, the parameter affected and the extent of the problem was identical for both 
engines. If the subject performed a normal cross-check between engines, it was hypo- 
thesized that it should have been easier to detect an out-of-tolerance parameter in one 
engine than in two engines because the displayed indications (location of the pointers or 
carets, level and rate of change of the numeric readout) would differ for the engines for 
that parameter. However, the benefit (if any) of adding the green "nominal range" 
bands (Display Conditions 3 and 5) might not be as great when a fault in a given para- 
meter occurs in a single engine as when the fault occurs simultaneously in both en- 
gines. This could result because visually comparing the position of the pointer or caret 
to the position of the green band for a given parameter may provide a more useful ref- 
erence for the expected range of a parameter than cross-checking the value of that para- 
meter on the second engine. 

5.4 Problem Onset Time 

The third variable manipulated in this study was Problem Onset Time: the amount of 
time it took for an engine problem to develop and manifest itself on the EICAS display(s). 
After a parameter value (or values) began to change when an engine problem was trig- 
gered, it could go outside the "nominal range" (if appropriate for the display condition), 
or exceed caution and warning limits, more or less rapidly. Problem Onset Time was 
defined as the amount of time from the problem trigger point to the time that the pointer 
and digital readout and box changed color and/or an alphanumeric alert message ap- 
peared, depending on the display condition. "Fast-onset" times were programmed to be 
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between five and nine seconds; "slow-onset" times were between 20 and 25 seconds. Pro- 
blem Onset Time was not independent of Display Condition, because the display changes 
that defined the end of the problem onset time (color changes and/or alphanumeric 
messages) appeared sooner for some display conditions than for others. For example, in 
the augmented EICAS conditions (Display Conditions 3, 4, and 5), a comparison of the 
current engine parameter values to the "nominal" values for those parameters provided 
by the engine model triggered color changes in the pointer and digital readouts when- 
ever an engine parameter value departed from the "nominal range." For Display Con- 
ditions 1 and 2, these same color changes did not occur until the parameter value ex- 
ceeded caution and warning limits. This meant that the pointer would have to travel 
farther from the baseline value in the latter display conditions before highly salient 
changes in the display would indicate a problem. For engine problems that develop very 
quickly, the addition of graphic information about normal parameter ranges may not be 
as useful in predicting engine problems, although it may be useful for overall problem 
monitoring. This problem and its implications are further discussed in the Data Ana- 
lysis and Results sections (7 and 8). 

For the primary independent variable, Display Condition, the order of training and test 
trials was completely counterbalanced to avoid or minimize training effects. The order 
of display conditions was counterbalanced by randomizing them, with the constraint 
that each of the five display conditions occur an equal number of times in the first, sec- 
ond, third, fourth, and fifth order of presentation for the ten subjects (twice in each ordi- 
nal position). A second constraint was that the sequential order of any two display con- 
ditions occur equally often. After constructing the order of display conditions for each 
subject, the display conditions were then paired with the five full-flight and ten take-off- 
only scenarios developed for the study (discussed below). For each subject, each display 
condition was paired with one of the full-flight scenarios (numbers 1 through 5) and two 
of the take-off-only scenarios (numbers 6 through 15); these pairings determined the 
three-flight trial blocks for each display condition for subject. Each display condition 
was paired equally often with each of the 15 scenarios, with the additional constraint 
that, for each three-flight trial block, each of the full-flight scenarios was paired with one 
of the take-off-only scenarios numbered 6-10 and one numbered 11-15. Furthermore, sce- 
narios 6-10 and scenarios 11-15 occurred equally often in the second and third position in 
the three-flight block. 

Scenarios 6-11 included engine problems, while scenarios 12-15 included non-engine 
problems, and this procedure roughly equalized the pairings of engine-problem and 
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non-engine-problem take-off-only scenarios with full flight scenarios. Table 3 gives the 
order of display conditions and scenarios paired with them for each of the ten subjects. 


Display Condition 


Basic El CAS 

Basic EICAS with alert messages 
Augmented EICAS with green bands 

Augmented EICAS with alert and "monitor parameter" messages 
Augmented EICAS with green bands & alert, monitor messages 


Basic EICAS with alert messages 
Basic EICAS 

Augmented EICAS with alert and "monitor parameter" messages 
Augmented EICAS with green bands & alert, monitor messages 
Augmented EICAS with green bands 


Augmented EICAS with alert and "monitor parameter" messages 
Augmented EICAS with green bands & alert, monitor messages 
Basic EICAS 

Augmented EICAS with green bands 

Basic EICAS with alert messages 


Augmented EICAS with green bands 

Augmented EICAS with green bands & alert, monitor messages 
Basic EICAS with alert messages 
Basic EICAS 

Augmented EICAS with alert and "monitor parameter" messages 


Augmented EICAS with green bands & alert, monitor messages 
Augmented EICAS with green bands 

Augmented EICAS with alert and "monitor parameter" messages 
Basic EICAS with alert messages 

Basic EICAS 


Basic EICAS 

Augmented EICAS with green bands 

Augmented EICAS with green bands & alert, monitor messages 
Augmented EICAS with alert and "monitor parameter" messages 
Basic EICAS with alert messages 


Basic EICAS with alert messages 
Basic EICAS 

Augmented EICAS with green bands 

Augmented EICAS with green bands & alert, monitor messages 
Augmented EICAS with alert and "monitor parameter" messages 


Augmented EICAS with alert and "monitor parameter" messages 
Augmented EICAS with green bands & alert, monitor messages 
Basic EICAS 

Basic EICAS with alert messages 

Augmented EICAS with green bands 


Augmented EICAS with green bands 
Basic EICAS with alert messages 

Augmented EICAS with alert and "monitor parameter" messages 
Basic EICAS 

Augmented EICAS with green bands & alert, monitor messages 


Augmented EICAS with green bands & alert, monitor messages 
Augmented EICAS with alert and "monitor parameter" messages 
Basic EICAS with alert messages 
Augmented EICAS with green bands 

Basic EICAS 


Full Flight 
Scenario 
# 


Take-off 

Scenario 

# 



Table 3. Display Condition and Scenario Order 
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Each of the three independent variables was counterbalanced with the others. Each full- 
flight scenarios included six non-normal events: four engine-related problems, and two 
non-engine problems. Each of the ten take-off-only scenarios included one non-normal 
event; six included an engine problem, and four included a non-engine problem. For 
each subject, there were a total of 26 engine-related problems: 20 during the five full- 
flight scenarios, and 6 during the take-off-only scenarios. Of the 20 engine problems in 
the full-flight scenarios, ten were one-engine problems and ten were two-engine pro- 
blems. Although the total number of engine problems were equally divided between one 
and two engine problems, and between slow- and fast-onset problems, this was not the 
case within a single full-flight scenario. At least one problem of each type (one versus 
two engine, slow- versus fast-onset) occurred in each scenario. Half of the one-engine 
problems overall and half of the two-engine problems overall had fast-onset times, and 
the other half had slow-onset times. Similar counterbalancing was carried out in the 
take-off only scenarios. Because there were only six problems per subject in these scena- 
rios, the Number of Engines and Problem Onset Time variables could not be completely 
counterbalanced within a subject, but they were counterbalanced across subjects. Table 
4 gives a list of all the engine problems (in chronological order) for each scenario. Table 
4 also includes the alphanumeric alert and "monitor parameter" messages associated 
with the problem, the time into the scenario that triggered the problem, the number of 
engines affected by the problem, the engine(s) affected, and the problem onset time. 

Because there were unequal numbers of one-engine versus two-engine problems, and 
slow-onset versus fast-onset problems in a given scenario, and scenarios were randomly 
paired with display conditions for different subjects, the result was an unequal number 
of data points per subject within a Display Condition level for one versus two engines, 
and for slow- versus fast-onset times. This was true even though the number of data 
points for these conditions were equal over all subjects combined. 

5.5 Simulation Methodology 

In the basic test paradigm, each subject flew a specified flight plan, and was asked to 
detect and identify non-normal events (engine and non-engine problems) as they oc- 
curred during the flight. During the test trials, both engine-related and non-engine- 
related conditions were presented. This was done in an attempt to keep the subject from 
focusing entirely on the graphical display of engine parameters and spending an inor- 
dinate amount of time scanning that portion of the display. Non-engine related pro- 
blems were chosen from the Boeing 767 alert message set and are shown in Table 5. 
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Table 4. Engine Problem Description By Scenario 









































































































































































Scenario 

# ’ T VP e 

Non-engine Probiem(s) 

EICAS Message(s) 

Levei 

Onset Time 


1- FF* 

Cargo door 

FWD CARGO DOOR 

Amber 

0 sec 

320 sec 


Left IRS failure 

LIRS FAIL 

Amber 

0 sec 

700 sec 

2- FF 

Aft cargo fire 

AFT CARGO FIRE 

Red 

0 sec 

210 sec 


Air conditioning pack failure 

R PACK OFF 

Amber 

0 sec 

680 sec 

3- FF 

Leading edge slat asymmetry 

LE SLAT ASYM 

Amber 

0 sec 

500 sec 


Low center hydraulic quantity 

CHYD QTY 

Amber 

0 sec 

600 sec 

4- FF 

Antiskid off 

ANTISKID OFF 

Amber 

0 sec 

220 sec 


Right generator off 

R GEN OFF 

Amber 

0 sec 

350 sec 

5- FF 

Cabin Altitude above 10,000 ft. 

CABIN ALTITUDE 

Red 

0 sec 

220 sec 


Left hydraulic pressure low 

L HYD SYS PRESS 

Amber 

0 sec 

600 sec 

6 - T/O * 

- 

- 

- 

- 

- 

7- T/O 

- 

- 

- 


. 

8- T/O 

- 

- 

- 


I - 

9- T/O 

- 

- 

- 

1 - 

- 

10- T/O 

- 

- 

- 

- 

- 

11 - T/O 

- 

- 

- 

- 

_ 

12 - T/O 

Stabilizer trim 


Red 

0 sec 


13- T/O 

Wheel well fire 

WHEEL WELL FIRE 

Red 

0 sec 

Vl**plus 15 kts 3 

14- T/O 

L AC bus off 

LAC BUS OFF 

Amber 

0 sec 

BBM M 

15- T/O 

Right IRS failure 

R IRS FAIL 

Amber 

0 sec 

BH3MM 


* FF=Full flight; T/0 =Takeoff only 
** assumes VI =130 kts 


Table 5. Non-Engine Problem Description by Scenario 


5.5.1 Flight Task 

In order to make the simulation as realistic as possible, the crews flew operational flight 
legs. The Research Flight Simulator (Research Cab) was used to create the flight deck 
environment and work patterns. A set of flight scenarios was developed for the study, 
using various airfields in Washington state. A total of 15 scenarios were generated. 

Five of these (scenarios 1-5) were extended "full flight" scenarios, approximately 10 to 12 
minutes in length, with a planned route from one airfield to another. Figure 11 shows 
the flight route for "full flight" scenario 1; the flight routes for the other four full-flight 
scenarios are given in Appendix A. The other ten scenarios (6-15) were "take-off-only;" 
they lasted less than two minutes. All take-off-only scenarios used Boeing Field in Seat- 
tle as the take-off field. 

For each of the five extended scenarios, six points were selected during the scenario at 
which non-normal events were triggered. These points were defined by time into the 
scenario, in seconds, and varied from scenario to scenario. The first non-normal event 
in each scenario occurred no soo ner than 120 seconds into the scenario; the other events 
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MWH RW32R 


1 . KMNH RW32R 

2. BATUM 

3. GGIAF3 

4. GGL0C3 

5. OLAKE (GEG) 

6. KG EG RW3 



followed at intervals that varied from 75 to 120 seconds. Four of the non-normal events 
in each scenario were engine-related and two were non-engine-related. For the ten take- 
off-only scenarios, a single trigger point for the non-normal event was defined by air- 
speed (in knots) relative to (plus or minus up to 40 knots). Table 6 shows the trial 

order/data sheet for subject 1. It includes the time- and Vj-based trigger points for all 

the non-normal events in all scenarios. Each full-flight scenario had an associated Air 
Traffic Control (ATC) script that provided a basic structure for a live controller who was 
permitted to improvise as the scenarios proceeded. The basic ATC script for full-flight 
Scenario 1 is given in Table 7; the scripts for the other full flight scenarios are in Appen- 
dix A. In order to provide a realistically high workload, the scenarios included a num- 
ber of flight route changes, intervening waypoints, and other attention-demanding man- 
euvers. 

During each flight trial the subject was required to manually fly the prescribed flight 
plan, respond to ATC communications, and respond to alerts. 

5.5.2 Response Task 

When a subject detected a non-normal condition, he was requested to depress a switch 
located on the left side of the control wheel (the lower microphone switch). This action 
was used to record the response time to detect the occurrence of a non-normal condition. 
Even though the switch press was artificial in relation to operational flight tasks, train- 
ing trials showed that the subjects did not have significant difficulty learning to respond 
in that manner. In addition, past test programs have shown that the subjects quickly 
learn to perform it and that it does not significantly affect flight task performance. 

After pressing the switch to indicate detection of a non-normal event, the subject was re- 
quired to verbally identify the problem. For engine-related problems, three types of infor- 
mation were required as part of the identification: the parameter involved (e.g., Nl), the 
engine exhibiting the problem (left, right or both), and the direction of the problem (high 
or low). Subjects were told that they could give this information in any order. The exper- 
imenter recorded the verbal response verbatim on the trial order/data sheet. If the sub- 
ject provided some but not all of the required information, the experimenter would 
prompt the subject for the information that had been omitted and record it verbatim if it 
was added. A response was counted as accurate only if all three types of information 
were given (with or without a prompt). 
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Table 6. Trial Order/Data Sheet (1 of 2) 
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Table 6. Trial Order/Data Sheet (2 of 2) 


























































































Mission Scenario 1 
KMWH/KGEG 


Cue 

ATC Message 

Crew calls for clearance prior 
to taxi 

Boeing 757 is cleared to the Spokane International 
Airport as filed. Climb and Maintain 8,000. Expect 
13,000 five minutes after departure. Leaving 3,000 turn 
right to heading 100 to intercept V2. Squawk 7571. 
Departure Control frequency will bel 20.85 

Crew calls ready to taxi 

Boeing 757, taxi to and hold short of Runway 32R at 
Taxiway A. 

Crew calls ready for takeoff 

Boeing 757, taxi into position and hold, Runway 32R. 

After aircraft is in position on 
the runway 

Boeing 757, cleared for takeoff. 

As the aircraft climbs through 
1,000 ft. AGL 

Boeing 757, contact Departure Control. 

Crew checks in on Departure 
Control 

Boeing 757, Roger. Say altitude. 

Crew reports altitude 

Boeing 757, Roger. 

Aircraft climbs through 7,000 
ft. MSL 

Boeing 757, contact Seattle Center on 126.1 

One minute after level at 
8,000 ft. MSL 

Boeing 757, radar contact, climb and maintain 13,000. 

One minute after level at 
13,000 ft. MSL 

Boeing 757, maintain airspeed at 320 knots or greater for 
sequencing. 

Aircraft passes D40/GEG 

Boeing 757, contact Spokane Approach Control on 
124.3. 

Crew contacts Spokane 
Approach Control 

Boeing 757, for traffic ahead, reduce speed to 210, then 
descend and maintain 5,000. Plan an ILS Runway 03 
approach. Intercept the ILS Runway 03 localizer via the 
10 DME Arc. Report crossing the 216° radial of the 
Spokane VOR. 

Crew reports crossing GEG 
216° radial 

Boeing 757, cleared for an ILS Runway 03 approach to 
the Spokane International Airport. Contact the Spokane 
tower 1 18.3 at OLAKE. 

Crew reports OLAKE 

Boeing 757, cleared to land, Runway 03. Wind 340 at 25 
with gusts to 35. RVR Runway 03 2,4000. Runway 03 
braking action reported poor by a 737. 

Aircraft slows to 60 knots 

Boeing 757, clear the runway at taxiway C, if able, then 
contact Ground Control on 121.9. 

Crew calls ground control 

Boeing 757, taxi to the terminal via taxiways M and H. 


Table 7. Air Traffic Control (ATC) Script for Flight Scenario 1 
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It was anticipated that the subject pilots would have diverse backgrounds, and have ex- 
perience with different response procedures for the various non-normal situations. In 
order to reduce the effect of this variation and to reduce the amount of time required to 
train the subjects in the simulator, the only response required for each of the non-nor- 
mals events was to push the "event recognized" switch and verbally identify the problem. 
Five seconds after the "event recognized" switch was pressed, the non-normal event was 
corrected without any further action on the part of the subject. If the problem was not 
detected within 15 seconds after it appeared in the EICAS display (as indicated by an 
alphanumeric message or change in color of the pointer/caret and digital readout), a 
"beep" sounded. Subjects had previously been instructed that if they heard the beep to 
press the lower microphone switch and verbally identify the engine problem type, en- 
gine(s) involved and direction of the problem. Those problems were also corrected with- 
in five seconds of the "beep" without any further action on the part of the subject. 

Each full-flight scenario included four engine problems and two non-engine problems. 
Six non-normal events in a 12-minute flight is greater than the number of problems ex- 
pected in normal flight operations. This number was selected in order to provide the 
subject with a sufficient quantity of engine-related non-normal events while maintain- 
ing a realistic minimum time period between occurrences. It was believed that the sub- 
jective data (ratings of the display conditions) would not be negatively affected by the 
event rate as long as enough time was available between the events for the subject to re- 
turn to normal operation. However, the higher rate of occurrence, and the fact that sub- 
jects knew that non-normal events would occur, could have affected the objective data. 
Previous research (Boucek, 1980) has shown that these factors can reduce the surprise 
and uncertainty factors and result in shorter response times than would be expected in 
operational situations. However, this should not have affected the validity of the results 
since only the relative differences between display concepts were of interest. 

5.5.3 Workload Measure 

In order to get an approximate measure of the workload involved in the test flights, the 
subjects were required to give a subjective workload rating twice during each of the ex- 
tended flight scenarios. At 180 and 540 seconds into each of the five full-flight scenarios, 
four "beeps" sounded and the experimenter announced "workload." Subjects had been 
instructed that when this occurred, they should respond with a workload rating by giv- 
ing a number from one to ten, based on the Bedford workload rating scale. The Bedford 
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workload rating scale is shown in Figure 12. Subjects received verbal instructions on 
how to use the Bedford scale before beginning the practice flights, and had a copy of the 
scale in the cab that they could refer to any time they wished. The experimenter record- 
ed the workload ratings when they were given. 


Workload Description Rating 



Figure 12. Bedford Workload Rating Scale 


5.6 Experiment Schedule and Procedures 

Each subject was run individually, and was scheduled for a five-hour block of time for 
the experiment. Table 8 presents an approximate schedule for a morning test session. 
Training, described in more detail below, consisted of a briefing on the study; an explan- 
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ation of the display conditions, the required responses, and the Bedford workload rating 
scale; a study period and a quiz on the values of engine parameters used in the study; 
and an explanation of the subjective rating questionnaires to be filled out after each trial 
block. This was followed by a familiarization period in the simulator. The subject was 
then introduced to the engine monitoring concepts and given hands-on response 
practice to non-normal conditions (including practice with the detection switch). 


TIME 

ACTIVITY 

7:00-8:00 

Simulator Checkout 

7:30-7:45 

Introduction to the Simulation Center 

7:45-8:45 

Briefing 

8:45-9:30 

Familiarization and Practice Trials 

9:30-9:45 

Break 

9:45-11:30 

Test Trials 

11:30-12:00 

Debriefing 


Table 8. Sample Subject Test Schedule 


Each subject then performed at least one full-flight practice flight, followed by four take- 
off-only practice flights. The test flights consisted of a total often take-off-only flights and 
five full-flight scenarios. Five blocks of test flights followed; each block included one full 
flight and two take-off-only flights. 

5.6.1 Briefing 

The experimental session for each subject began with an introduction to the Flight Sim- 
ulation Center. This introduction included a brief description and walk through the 
facility. Following this introduction, the subject was briefed in a room adjacent to the 
simulator. Each subject was given a brief summary of the background and purpose of 
the study, and why his input was sought. For each display condition, a series of line 
drawings was used to show what the upper and (if appropriate) lower EICAS screen 
would look like for each of several circumstances: all parameters normal; one parame- 
ter in the caution or warning range; and one parameter outside the nominal value 
range, but not yet into the caution or warning range. The drawings included examples 
of both round-dial and vertical -scale parameters. Each line drawing was appropriately 
colored to show the changes in coding that would occur, and labelled to indicate the spe- 
cific condition that the drawing represented. 
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The subject was then briefed on the operation of the simulator, including its instrumen- 
tation and how it differed from a standard 757 flight deck. He was then given a list of 
"Critical Engine Parameters," shown in Table 9. This list included the normal, caution, 
and warning (both high and low) ranges for all the engine parameters shown on the 
EICAS displays. The subject was asked to memorize the list and told that he would be 
given a quiz on the information. The subject was allowed to take as much time as he 
wanted to study the list, after which he completed a fill-in-the-blank quiz. Each subject 
completed the quiz with a minimum of 96 percent accuracy. 


N1 Limits: 


EGT Limits: 


N2 Limits: 


Oil Pressure: 


Oil Temperature: 


Oil Quantity: 


Caution (low) 
Normal 

Caution (high) 
Warning (high) 

Caution (low) 
Normal 

Caution (high) 
Warning (high) 

Caution (low) 
Normal 

Caution (high) 
Warning (high) 

Warning (low) 
Caution (low) 
Normal 

Caution (high) 

Caution (low) 
Normal 

Caution (high) 
Warning (high) 

Warning (low) 
Normal 


0 to 20% 

20 to 100% 

100 to 110% 
above 1 1 0% 

below 250°C 
250 to 600°C 
600 to 800°C 
above 800°C 

0 to 60% 

60 to 100% 

100 to 110% 
above 100% 

at or below 70 PSI 
70 to 75 PSI 
75 to 160 PSI 
above 160 PSI 

below 40°C 
40 to 163°C 
163 to 177°C 
above 177°C 

below 4.0 qts 
above 4.0 qts 


Table 9. Critical Engine Parameter List 
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The Bedford workload rating scale (shown earlier in Figure 12) was then explained to 
the subject, and he was given an opportunity to study the scale and ask any questions 
about it. Each subject had a copy of the rating scale with him for reference throughout 
the practice and test trials. 

The objective response measures were then explained to the subject. Both speed and ac- 
curacy of responses were emphasized. Because response time was recorded when the 
lower microphone switch was pressed, each subject was instructed to press that switch 
as soon as he detected a non-normal event (either engine or non-engine), and then to 
give the verbal description of the problem. As noted earlier, each subject was instructed 
to give three pieces of information about each engine problem: the parameter involved, 
which engine(s) exhibited the problem, and the direction (high or low) of the problem. 
The subject was told that he could give the information in any order. For non-engine 
problems, he was told to identify the nature of the problem; in essence, this meant read- 
ing the caution or alert message on the upper EICAS screen. Each subject was then 
shown samples of the subjective rating questionnaires to be filled out for each display 
condition. Finally, each subject was given a questionnaire to fill out about his back- 
ground and flight experience. The questionnaire is shown in Appendix B, and the data 
from this questionnaire is summarized in Table 1. Each subject was encouraged to ask 
questions during all parts of the briefing. The subject then moved to the simulator, 
where he had hands-on practice with the alerting system, engine displays, response 
switch, and data collection procedures. 

5.6.2 Familiarization and Practice Trials 

Each subject flew at least five familiarization/practice flights in the simulator before test 
trials began. The first flight was a 10 to 12 minute "full flight" using one of the five dis- 
play conditions. The features associated with the given display condition were explained 
at the beginning of the flight. The subject was instructed to make all required responses, 
and any mistakes or omissions were corrected. The copilot/observer in the right seat 
provided instruction about flying the simulator, demonstrated response procedures, and 
noted any difficulties that the subject had in adjusting to the flight characteristics of the 
simulator. Most subjects showed adequate flight proficiency at the end of the full-flight 
scenario; those who did not were allowed to fly another complete or partial full-flight 
scenario with the same display condition. Then each subject flew four additional take- 
off-only scenarios, each with one of the four remaining display conditions not used in the 
initial full-flight scenario. At the beginning of each take-off-only flight the features of 
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the display condition specific to that practice trial were reviewed, and the subject was 
asked if he had any questions. There were few questions during the practice trials, and 
they were all answered completely. Objective data collection procedures were carried 
out during all practice trials, and workload ratings were also recorded to familiarize the 
subjects with making those responses. Subjective questionnaires, however, were not 
filled out during the practice period. The order of the practice-trial scenarios and the 
order of the display conditions paired with those scenarios was randomized and com- 
pletely counterbalanced across subjects. 

5.6.3 Test Trials 

After all practice trials were completed, test (data collection) trials blocks began. Sub- 
jects first flew a full-flight scenario, followed by two take-off-only scenarios; the same dis- 
play condition was used in all three scenarios. The specific features of the appropriate 
display condition were reviewed at the beginning of the three-scenario trial block. At the 
completion of each block of test flights, each subject filled out a questionnaire to provide a 
subjective evaluation of the display condition used in that block. 

5.6.4 Debriefing 

At the end of the full test session, each subject filled out a final questionnaire to provide 
comparisons among the display conditions, and to allow the subjects to make more open 
ended comments. The experimenter conducted a debriefing and answered any ques- 
tions. 

6. DATA COLLECTION 

The variables that were not being tested were held constant or controlled so that they 
would not bias or confound the results. During the flight task, aircraft noise of approxi- 
mately 75 db was used to mask the uncontrolled noise that might occur around the cab. 
The ambient lighting was kept relatively low to permit the use of the outside visual 
scene. ATC communication and aural alerts were presented approximately 8 db above 
the ambient noise and held constant for all trials. The ATC messages were approxi- 
mately the same for each subject. All subjects received the same instructions and train- 
ing to avoid experimental bias. 
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Whenever the performance of the same task is measured under several different treat- 
ment conditions, as in the present experiment, learning and/or fatigue effects from ear- 
lier trials may influence the performance on later trials. Care was taken to prevent 
these carry-over effects from differentially affecting the performance measures for the 
different treatment conditions. This was accomplished by counterbalancing the order 
which the subjects received the treatment conditions. The subjects were informed before 
each flight which display condition would be used for that flight. 

The data that was collected during the display evaluations fell into two categories: ob- 
jective data (response time to detect non-normal events and accuracy of identifying 
them) and subjective data (rating and comment questionnaires and subjective workload 
measures). 

6.1 Objective Data 

The objective measures that were used during the study concerned the subjects' recogni- 
tion of non-normal events and the acquisition of relevant information from the EICAS 
displays. 

The measures that were used to provide insight into the issues of information transfer 
and the display effectiveness were detection time (response time) and identification ac- 
curacy (response accuracy). Detection time was defined as the time between the trigger 
time for a non-normal event (engine or non-engine problem) and the time that the sub- 
ject pressed the lower microphone switch to indicate that the event had been detected. 
Response time was automatically recorded when the switch was pressed. As noted ear- 
lier, however, this definition included an unequal amount of time between the problem 
trigger and the full problem onset (the point in time when a color change or alphanu- 
meric message appeared on the display) for the different display conditions. A color 
change (in the pointer and digital readout and box) occurred in Display Conditions 3, 4, 
and 5 (augmented EICAS conditions) when the value of a displayed parameter went out- 
side the nominal range, whether or not that range was graphically depicted by green 
bands. The same color changes did not occur in Display Conditions 1 and 2 (basic 
EICAS) until the value of the parameter exceeded caution and warning limits. Similar- 
ly, alphanumeric engine parameter alert messages first appeared in Display Conditions 
4 and 5 when the value of the parameter went outside the nominal range, but did not ap- 
pear in Display Condition 2 until the value of the parameter exceeded caution and warn- 
ing limits. Alphanumeric messages did not occur for engine problems in Display Con- 
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ditions 1 and 3. Therefore, if color changes were a critical cue for detecting an engine 
parameter problem on the EICAS displays, then the average response times for the five 
display conditions should have been, in increasing order, for Display Conditions 3, 4, and 
5, followed by Display Conditions 1 and 2. If the appearance of an alphanumeric engine 
parameter message (alert or "monitor parameter") was a critical cue, then the response 
times in increasing order should have been for Display Conditions 4 and 5 followed by 
Display Conditions 2, and then by Display Conditions 1 and 3. 

As indicated earlier, a limit was put on the amount of time for the subject to respond to a 
non-normal event (engine or non-engine). If the subject did not respond within 15 sec- 
onds after a color change or alphanumeric engine parameter alert or "monitor parame- 
ter" message was presented, then a beep sounded and the subject was instructed to ver- 
bally identify the problem. In that case, the problem onset time plus 15 seconds was re- 
corded as the response time for that non-normal event. 

Plight task performance was not assessed during this study. Because data was collected 
in both full-flight and take-off only scenarios, there was only one traditional flight perfor- 
mance variable that could have been measured across all data trials: heading accuracy. 
Because the take-off only scenarios were of such short duration, however, it was decided 
that data about heading accuracy would not provide much useful information. 

6.2 Subjective Data 

Subjective data was gathered throughout the experiment in order to assess the subjects' 
judgements of the usefulness and interpretability of the engine monitoring display con- 
cepts. There were three formal methods of gathering subject opinion concerning the 
monitoring display. Following the block of three flights for each display condition, each 
subject was given a short questionnaire to evaluate that display condition while the ex- 
perience with it was fresh. Subjects rated the ease and speed of interpreting engine 
parameter information (overall, and in both the round-dial and vertical-scale formats) 
on the EICAS displays. At the end of the test session each subject filled out a question- 
naire to rank order the five display conditions according to ease of use. They also rated 
the overall usefulness and ease of interpreting the three features added to the basic 
EICAS display and examined in this study: alphanumeric alert (caution and warning) 
messages for engine parameters, alphanumeric "monitor parameter" messages, and 
green "nominal range" bands. Three open-ended questions were also included to elicit 
any positive and negative opinions about the display features, and any other comments. 
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The questionnaires used to evaluate each of the display conditions with a summary of 
the responses are given in Appendix C. The final questionnaire used to rank them and 
evaluate the features and a summary of responses are also given in Appendix C. 

6.3 Workload Data 

In addition to the opinion data about the display formats and features, the subjects were 
asked to provide subjective ratings of their workload during each of the full flights. The 
Bedford workload-rating scale was used because of its non-intrusive nature. A replica of 
the Bedford rating scale was available to them for reference during the practice and ex- 
perimental sessions. 

7. DATA ANALYSIS AND RESULTS 

7.1 Objective Data 

The objective data were analyzed using an analysis of variance program designed for 
repeated measures. Experimental errors and problems in data recording resulted in 
missing data for three full-flights and one take-off-only flight across all subjects for the 
response time dependent variable. In the sections that follow, data will be reported as 
means across subjects. Means reported in the text and shown in the figures represent 
all data points, uncorrected for missing data or unequal number of observations. 

In order to perform statistical analyses for the repeated measures design used in this 
study, equal numbers of data points were required for each level of each independent 
variable, and each combination of independent variables. There were equal numbers of 
data points per subject for the Number of Engines and Problem Onset Time variables. 
However, because the Number of Engines and Problem Onset Times were unevenly dis- 
tributed within scenarios, which were then randomly paired with Display Conditions, 
there were not equal numbers of observations in all Number of Engines/Problem Onset 
Time/Display Condition cells. Because of these problems with missing and unequal ob- 
servations, the analyses were done separately for the Number of Engines/ Problem Onset 
Time and the Display Condition variables. 

There were 26 possible data points per subject for each of the dependent variables: re- 
sponse time to detect a non-normal engine event (in seconds) and identification accur- 
acy. Thirteen of these data points were for single engine problems, and 13 were for two- 


40 


engine problems. Similarly, 13 of the data points were for fast-onset problems, and 13 
were for slow-onset problems. Missing observations (a total of 9 data points across all 
subjects) were replaced by the appropriate cell mean for the subject and a repeated- 
measures ANOVA was then performed on these data. 

Figures 13 and 14 show the average response times for Number of Engines and Problem 
Onset Time. Significant effects were found for Problem Onset (F=19.25, df=l,9, p<.005) 
and for Trials (F=16.51, df=6,54, p<.001). Responses were faster for "fast-onset problems" 
(9.0 sec) than for "slow-onset problems" (14.59 sec), and there was a significant amount 
of variability in responding to different problems within different scenarios. There was 
no effect for Number of Engines. Average response times for engine versus non-engine 
problems are shown in Figure 15. The average response time for non-engine problems 
was much faster then for engine problems because the only requirement for non-engine 
problems was that the subjects read the caution or warning message. 

Figures 16 and 17 show the average identification accuracy for Number of Engines and 
Problem Onset Time. No significant effects were found. For Number of Engines, the 
accuracy of problem identification was slightly higher for two-engine problems (91%) 
than for one-engine problems (87%). Accuracy was virtually identical for slow-onset 
(89%) versus fast onset (88%) problems. Figure 18 shows that identification accuracy 
was 100% for non-engine problems. 

A one-way analysis of variance for Display Condition was carried out with the response 
time and identification accuracy data for each subject. For response time, the mam ef- 
fect of Display Condition was highly significant (F=24.38, df=4,36, pc.OOOl). Figure 19 
shows that average response times decreased in order from Display Condition 1 to Dis- 
play Condition 5 (18.56, 13.51, 12.19, 7.75, and 6.73 sec, respectively). But recall that re- 
sponse time was confounded with the amount of time that it took for a an engine pro 
blem to trigger a color change or message in the display. For color changes, this time 
decreased from Display Condition 1 to Display Conditions 2, 3, 4 and 5, for messages, it 
decreased from Display Condition 1 to Display Condition 2, and from Display Condition 3 
to Display Conditions 4 and 5. 

No effects were significant for identification accuracy. Identification accuracy is shown 
in Figure 20. Accuracy was highest for Display Condition 2 (98%), followed by Display 
Conditions 1 and 3 (both 92%), and then by Display Conditions 4 and 5 (82% and 80%, re- 
spectively). 
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Figure 17. Average Identification Accuracy by Problem Onset Time 
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Figure 19. Average Response Time (sec) by Display Condition 
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Figure 20. Average Identification Accuracy by Display Condition 
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7.2 Subjective data 


Ratings collected about each of the display conditions at the end of each trial block 
were analyzed separately from the overall ratings and ranking of display conditions. 
A one-way analysis of variance for repeated measures was applied to all the sub- 
jective rating and ranking data. There were no missing subjective data. 

At the end of each data collection block, subjects rated the overall ease of interpre- 
tation, the ease and speed of interpreting round-dial format information, and the ease 
and speed of interpreting vertical-scale information for the display condition used in 
that block. All ratings were on a scale of 1 (extremely easy) to 5 (moderately easy). 

For purposes of analyzing and graphing the results, these ratings were transformed 
so that higher numbers represented more preferred conditions (5 = extremely easy 
and 1 = moderately easy). All means reported in the text and figures represent the 
transformed ratings. 

Figure 21 shows the average ratings for overall ease of interpretation for each display 
condition. Subjects rated Display Condition 5 easiest, followed by Display Conditions 
4, 3, 2, and 1 (ratings of 4.0, 3.2, 3.0, 2.9, and 2.8, respectively). A one-way analysis of 
variance showed that the overall effect of Display Condition was significant (F=3.68, 
df=4,36, pc. 05). Tukey test comparisons were then made between pairs of individual 
means. The only paired comparison that showed a significant difference was 
between Display Condition 5, which included engine parameter alert messages, 
"monitor parameter" messages and green bands; and Display Condition 1 (basic 
EICAS). There were no pair-wise comparisons that showed a significant difference 
on the ratings of ease or speed of interpretation of round-dial or vertical-scale 
information. 

After all trial blocks were completed, subjects also rank-ordered the Display Condi- 
tions for ease of use. Figure 22 shows the average rankings for each of the display 
conditions. In order of preference, the rankings were Display Conditions 2, 4, 5, 3, 
and 1. A one-way analysis of variance on the rankings showed a significant differ- 
ence for Display Condition (F=5.29, df=4,36, p<.005). Subsequent Tukey tests for pair- 
wise comparisons showed that the rankings for Display Conditions 2, 4, and 5 were 
each significantly greater than the rankings for Display Conditions 1 and 3. After all 
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Figure 22. Average Rankings of Display Conditions for Ease of Use 
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trial blocks were completed, subjects also rated the usefulness and ease of interpretation 
of the three features added to the displays (alphanumeric engine parameter alert 
messages, alphanumeric "monitor parameter" messages, and green "nominal range" 
bands). 

These ratings were on a scale of 1 to 5 for usefulness (not at all useful to very useful) or 
and interpretability (very difficult to interpret to very easy to interpret). Figure 23 shows 
the average ratings of "usefulness" for the three features. Alphanumeric alert mess- 
ages were rated most useful (4.6), followed by the "monitor parameter" messages (4.2), 
and then by the green bands (2.7). The ratings were analyzed using the BMDP 2V one- 
way analysis of variance program. There was a significant effect of feature type 
(F=12.37, df=2,18, pc.OOl). Tukey test comparisons between means showed that the rated 
usefulness of both types of alphanumeric messages (engine parameter alert (caution/ 
warning) messages and "monitor parameter" messages) was significantly higher than 
the rated usefulness of the green "nominal range" bands; no other paired comparisons 
were significant. 

Figure 24 shows the average ratings of "ease of interpretation" for the three features. 

The alphanumeric alert messages were rated easiest to interpret (4.8), followed by the 
"monitor parameter" messages (4.2), and then by the green bands (3.3). There was again 
a significant effect of feature type (F=5.94, df=2,18, p=.01). Tukey tests showed that the 
rated ease of interpretation of the caution/waming alert messages was significantly 
higher than the rated ease of interpretation of the green bands; no other paired compar- 
isons were significant. Subjects made a variety of responses to the open-ended questions; 
their responses are summarized in Table 10. 


7.3 Workload Data 

The average Bedford Scale workload ratings for the five display conditions are shown in 
Figure 25. Workload ratings were almost identical across the conditions, which sup- 
ports the hypothesis that differences in workload were not responsible for differences in 
the objective or subjective dependent variables. 
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In general, what did you like about the features associated with the 
augmented EICAS displays? 

Parameter and monitor messages. (2 subjects) 

Easy and quick problem analysis. The oil temperature and oil pressure 
messages were very helpful. 

Good for trend monitoring only. 

Monitor and parameter messages were a nice addition to the EICAS. 

More useful information. 

Called attention to the exact problem or impending problem. 

Engine parameter messages were good for alerting to a particular problem. 
Catches your eye. 

Allows a more rapid recognition of change. 


In general, what did you dislike about the features associated with the 
augmented EICAS displays? 

Monitor messages were somewhat distracting in that you had to follow up the 
message by looking at the particular gauge. 

Normal range of parameter not as useful as EICAS message. 

Green bands seemed unnecessary. 

Green bands of minimum use. 

Requires more observation to detect changes. 


If you have any additional comments, please include them here. 

Possible difficulty interpreting messages with multiple messages on EICAS 

Quantitative data will most likely be more useful in this evaluation than the 
questionnaires. 

I found myself concentrating more on the EICAS display than is realistic. 
Otherwise the information presented during malfunctions was useful. 

Lower EICAS screen should receive more augmentation because it is 
scanned less. 

I like EICAS with parameter and monitor messages. 

Bar displays are more recognizable when changes occur in the parameters 
than dial displays. 


Table 10. Summary of Responses to Open-Ended Questions on Final 

Questionnaire. 
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Figure 25. Average Bedford Scale Workload Ratings by Display Condition 


8. DISCUSSION AND CONCLUSIONS 

In general, both the objective and subjective data showed that adding alphanumeric 
alert (caution and warning) messages to the basic EICAS displays, similar to those used 
for other systems problems, decreased response time. The data also indicated that the 
alert messages were the added feature most preferred by the subjects. Adding alpha- 
numeric messages to "monitor" a parameter that deviated from a nominal range also 
decreased response time, and was the added feature subjects preferred next. Adding 
green "nominal range" bands did not appear to decrease response time, and was the 
least-preferred feature. The fastest average response time was associated with Display 
Condition 5, which included all three added display features and was ranked highest by 
the subjects; the slowest average response time was associated with the basic EICAS 
display format, which included none of the added features and was also least preferred 
by the subjects. 
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8.1 Objective data 


Technical difficulties with the data collection procedures made this data difficult to 
interpret. As noted earlier, response time was measured from when a problem was 
triggered. But the display conditions differed in the amount of time it took for certain 
display cues (color changes, appearance of alphanumeric messages) to indicate that 
a problem had occurred. These circumstances made it impossible to make direct 
comparisons among the display conditions on the objective measures, because both 
response time and identification accuracy were sensitive to the amount of time 
available. An inspection of the means for response time across display conditions 
shows that performance was better, as expected, for the conditions that provided the 
earliest display cues: Display Conditions 3, 4, and 5 were better than Display Condi- 
tions 1 and 2. Note that this does not support one possible hypothesis that subjects 
would detect a developing engine problem on the basis of changes in the digital read 
out and/or pointer movement, before a color change occurred or a message appeared. 
Even though the timing of these readout and pointer cues was identical for all display 
conditions, the average response times were not. 


Comparisons between specific display conditions, however, allow the following con- 
clusions to be drawn. Even though the differences between the display conditions 
discussed below were not statistically significant, it can be seen from Figure 19 that 
adding alphanumeric engine parameter alert or "monitor parameter" messages 
decreased response time. Display Conditions 1 and 2 were identical in the timing of 
the color changes in the pointers/carets and digital readouts that occurred when an 
engine parameter exceeded caution and warning limits. But in Display Condition 2, 
an alphanumeric caution or warning message appeared concurrently with the color 
change, and the response time was faster than for Display Condition 1. Similarly, 
Display Conditions 3 and 5 differed only in that alphanumeric "monitor parameter" 
messages accompanied the color change cues that occurred when an engine para- 
meter departed from the "nominal range" band generated by the engine model. 

Again, average response time was faster for Display Condition 5, which included 
alphanumeric messages, than for Display Condition 3, which did not. These compar- 
isons show that the effect on response time holds whether or not green bands graph- 
ically depict the nominal range; it also holds whether the alphanumeric messages 
are caution and warning messages similar to those used for other systems, or 
messages to "monitor" a given parameter. Presenting an alphanumeric message, 
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especially one that is similar in location and color to caution and warning alert 
messages for other systems, appears to decrease the amount of time it takes to detect 
a problem that involves an engine parameter presented on the EICAS displays. 

There does not, however, appear to be an effect for adding green bands that graph- 
ically depict the range of nominal values generated by the engine model. The only 
direct comparison that is appropriate for this hypothesis is between Display Con- 
ditions 4 and 5 (Figure 19). This comparison indicates that the addition of green 
bands does not make much difference in the average response time. The lack of a 
substantial difference in response time may be masked by the overall effectiveness of 
adding alphanumeric messages to the display. There is no pair of display conditions 
that would allow an unambiguous comparison of the effects of green bands alone, 
because the appropriate pair of conditions (Display Conditions 1 and 3) differed in the 
point at which other changes occurred in the display (color cues changed when the 
value of the parameter departed from the nominal range in Display Condition 3, but 
only when the value of the parameter exceeded a caution or warning limit in Display 
Condition 1). 

In general, although differences in response accuracy were not statistically different, 
the data for response accuracy support the conclusions drawn by examining the re- 
sponse time data. Figure 20 in section 7 shows the accuracy of verbal identification 
responses for each display condition. Recall that a response was scored as accurate 
only if the subject correctly provided all three pieces of information about the problem: 
which parameter was involved, in which direction the deviation occurred, and which 
engine or engines were affected. Accuracy was higher in Display Condition 2 (98%), 
which included alphanumeric engine parameter alert messages than for the 
baseline EICAS format without messages (Display Condition 1, 92%). But accuracy 
in Display Condition 3 (92%), which added green bands to depict the range of nominal 
values, was identical to the baseline. Identification accuracy was lower than the 
baseline in Display Conditions 4 (82%) and Display Condition 5 (80%). In both these 
display conditions, the first alphanumeric message that appeared was one that 
advised the subject to monitor a parameter as it departed from the nominal range. 
These "monitor parameter" messages differed from the caution and warning 
messages that appeared first in Display Condition 2. The caution and warning 
messages in general included information about each of the three required pieces of 
information, e.g.J'h EGT HIGH." Because of constraints on the number of letters 
that could be used in an alphanumeric message, the "monitor parameter" messages 
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did not always contain the same amount of information, e.g., "MONITOR L EGT". 
This meant that subjects generally only had to read the alphanumeric message to 
obtain all the required pieces of information in Display Condition 2, but often had to 
cross-check the graphic display to do so in Display Conditions 4 and 5. In addition, 
subjects often read the "monitor parameter" message before they realized that it did 
not contain all the information that was required. They then had to cross-check the 
information on the graphical portion of the display. However, five seconds after the 
response switch was hit, the problem resolved itself, and the information may no 
longer have been available. Display Condition 2, with the addition of caution and 
warning alert messages only, had the most similar, highly salient display cues 
(alphanumeric caution and alert messages) for both engine and non-engine 
problems, and this may have increased accuracy by reducing response uncertainty. 
In addition, alphanumeric messages accumulated in all the display conditions that 
included messages until the problem was resolved. This meant that Display 
Conditions 4 and 5 could have twice as many messages displayed (both alert and 
"monitor parameter" messages) as Display Condition 2 by the time an engine 
problem was fully developed. This increase in the number of displayed messages 
could also have increased response uncertainty and thus decreased the response 
accuracy of Display Conditions 4 and 5 relative to Display Condition 2. 

As shown earlier in Figure 14, there was a highly significant effect of Problem Onset 
Time on response time; this was expected because problems with a slow-onset time 
took longer to develop and appear in the EICAS displays, and thus longer to detect. 
The effect of Number of Engines on response time, however, was not significant. It 
took just as long to detect a problem that occurred in two engines as it did to detect a 
problem that occurred in one engine (Figure 15). It was hypothesized that the effect 
of adding green "nominal range" bands to the graphical parameter displays might 
have a greater effect for two-engine problems than for one-engine problems. This 
hypothesis, however, was not supported by the data. The interaction between Display 
Condition and Number of Engines was not significant. Figure 26 shows that 
responses were marginally faster for two-engine than for one-engine problems for 
the display conditions that included green bands (Display Conditions 3 and 5), and 
almost equal or marginally slower in the remaining conditions, but the interaction 
was not significant. Identification accuracy was equal or higher for the two-engine 
problems in all the display conditions (Figure 27). 
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Figure 26. Average Response Time (sec) for Display Condition 

by Number of Engines 


■ ONE ENGINE □ TWO ENGINES 



BASIC EICAS 

BASIC EICAS 

AUGMENTED 

AUGMENTED 

AUGMENTED 

= 


+ ALERT 

EICAS + GREEN 

EICAS+ALERT, 

EICAS+GREEN 

I 


MESSAGES 

BANDS 

MONITOR 

BANDS+ALERT, 

- 




MESSAGES 

MONITOR MSGS 

- 


Figure 27. Average Identification Accuracy for Display Condition 

by Number of Engines 
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8.2 Subjective data 


The subjective data generally supported the conclusions drawn from the objective data. 
Subjects rated the overall ease of use of each of the five display conditions immediately 
after the test flights that incorporated the display condition, and then rank-ordered the 
five display conditions after all test flights were completed. When subjects rated the dis- 
play conditions immediately after the trial block, the ratings were fairly similar for four 
of the five display conditions, and the ratings increased in order from Display Condition 
1 to Display Condition 5. Slight increases in the ratings were associated with the addi- 
tion of either alphanumeric messages or green bands alone (Display Conditions 2 
through 4); a significantly higher rating was associated with the addition of bQth mess- 
ages and green bands (Display Condition 5). However, when forced to rank-order the five 
display conditions, subjects ranked highest the display conditions that included alpha- 
numeric engine parameter messages (both alert and monitor parameter messages), 
with or without the addition of green nominal range bands (Display Conditions 2, 4, 
and 5). These were followed by the display condition that included green bands alone 
(Display Condition 3); least preferred was the basic EICAS (Display Condition 1), which 
did not include either messages or green bands. Ratings of the usefulness and ease of 
interpreting these features agreed with the relative rankings of the display conditions: 
engine parameter alert messages, "monitor parameter" messages, and green bands, re- 
spectively, were rated most to least useful and easy to interpret. 

Answers to the open-ended questions at the end of all session, summarized earlier in 
Table 10, were consistent with these findings. Subjects mentioned that the alphanumer- 
ic engine parameter messages were useful additions to the display. One subject noted 
that the monitor messages were somewhat distracting and required a follow-up look at 
the graphic display of the parameter. When asked what they disliked about the displays, 
some subjects responded that the green bands and the "normal range" information 
were not as useful or necessary. Thus, in general, subjects found unambiguous alpha- 
numeric messages about engine parameters a useful addition to the displays; they did 
not find the nominal range information, at least as implemented in this study, as useful. 

8.3 Recommendations 

One conclusion that may be drawn from both the objective and subjective data is that 
adding alphanumeric alert messages for problems involving engine parameters would 
aid in their detection and identification. Subjects made it clear that they found the al- 
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phanumeric engine parameter alert messages both useful and easy to interpret; they 
also responded faster and with greater accuracy when such messages were included in 
the display. Thus, one recommendation made as the result of this study would be to in- 
tegrate engine parameters into the caution and warning alert system. That would in- 
volve adding caution, warning and status messages to the current list of messages 
available for other systems, and would provide an aural alert for all caution- and warn- 
ing-level engine parameter problems. This change would provide the major advantage 
of consistency of coding across all systems problems. In addition, the alphanumeric en- 
gine parameter messages should be coded so that all the crucial information about the 
problem is included in the message: the engine and parameter that are involved, and if 
appropriate, the direction of the problem. This would make easily-assimilated informa- 
tion about engine-parameter problems complete in a single location, so that cross-check- 
ing with graphics would not be required. This is especially important in conditions in 
which the information is out of the pilots' primary scan (for example, on the lower 
EICAS screen) and for which a timely and rapid response is required. 

Conclusions about the utility and interpretability of the green bands to depict nominal 
range values and the "monitor parameter" messages, and recommendations about their 
inclusion are less certain. These features were added to the display to enable the sub- 
jects to monitor and detect potential problems in engine parameters as they develop, but 
subjects had mixed responses to them. It is important to note that any conclusions must 
be based on the features as they were implemented and tested in this study. Subjects 
clearly preferred the addition of alphanumeric "monitor parameter" messages to hav- 
ing no messages at all, and the addition of these messages decreased the time required 
to detect a developing engine parameter problem. As discussed above, however, the 
wording of these messages should be clear and unambiguous. Further work should be 
done to determine the optimum wording and timing of such messages. The addition of 
green bands to indicate the range of nominal values for a given parameter, as imple- 
mented in this study, was not as useful or well received. Part of the problem may have 
been that the bands were relatively small and thus less easy to detect than they would 
have been if implemented in the larger vertical tape format used in the 747. It was 
hypothesized that subjects would use the location of the green bands to help determine 
whether the value of a parameter that was departing from the nominal range was high 
or low. An inspection of the verbal responses given by the subjects, however, indicated 
that a common form of response was, for example, "right EGT increasing". This indi- 
cates that in many cases, subjects were using the direction of movement of the pointer or 
caret for a parameter, and not its relative location, to determine whether the value of a 
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parameter was high or low. Since relative movement appears to be a more salient and 
easily interpreted cue than relative location, its use as cue for monitoring engine para- 
meters should be further explored. It should also be noted that the subjects in this study 
were all captains; first officers are usually responsible for monitoring systems problems, 
including engine problems. The subjects were not used to keeping the EICAS displays 
in their scan pattern, and many of them commented on the difficulty of doing so while 
they were manually flying the aircraft (another infre(juent task). Under circumstances 
where a first officer was more closely monitoring the EICAS displays without the work- 
load of flying, the results obtained in this study may have been different. 

Including task-oriented information in a display is clearly the objective of any good hu- 
man factors design process. Based on the results of this study, the integration of engine 
parameter information into the EICAS caution and warning system appears to be bene- 
ficial and desirable to line and training pilots. Incorporating an engine model that can 
compare current engine parameter values to those that are normal for the given flight 
phase and engine history may be an improvement on a system that only flags problems 
after they have reached a caution or warning level. But the specific circumstances un- 
der which this addition may be beneficial, as well as the best format for this additional 
task-oriented information must be further investigated. The addition of alphanumeric 
messages to monitor parameters that deviate from a normal range appears to be more 
beneficial and more preferred than the addition of a graphic depiction of the normal 
range as implemented in this study. The incorporation of any additional information 
into a display must carefully take into account the already existing display suite and 
coding conventions. Further studies should explore the most effective implementation 
q]P 0nginc parameter monitoring and alert information consistent with the overall flight 
deck design philosophy. 
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APPENDIX A 

Flight Routes and ATC Scripts for Full-Flight Scenarios 


This appendix includes the planned flight routes and the associated Air 
Traffic Control scripts for the full-flight scenarios used in the study. 
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MWH RW32R 


1. KMNH RW32R 

2. BATUM 

3. GGIAF3 

4. GGL0C3 

5. OLAKE (GEG) 

6. KG EG RW3 



4 


GEGALW 


f 1 4 

KGEG / KALW 



TRINA 





Company Route ~ FTC 01 



cf!3r 


NOLLA 


KB FI / KBFI 



SEA A -.. 
LACRE 


1. KBFT RW13R 

2. LACRE 

3. VAMPS 

4. PAE 

5. cfl3r 

6. NO [.LA 


VAMPS 







1. KMWH RW32R 

2. EPH 

3. EAT 

4. <Juv;il 

5. cfi3r 

6. HOLLA 

7. ILS13R 




ATC Script - Mission Scenario 1 
KMWH/KGEG 


Cue 

. ATC Message 

Crew calls for clearance prior 
to taxi 

Boeing 757 is cleared to the Spokane International 
Airport as filed. Climb and Maintain 8,000. Expect 
13,000 five minutes after departure. Leaving 3,000 
turn right to heading 100 to intercept V2. Squawk 
7571. Departure Control frequency will be 120.85 

Crew calls ready to taxi 

Boeing 757, taxi to and hold short of Runway 32R at 
Taxiway A. 

Crew calls ready for takeoff 

Boeing 757, taxi into position and hold, Runway 32R. 

After aircraft is in position on 
the runway 

Boeing 757, cleared for takeoff. 

As the aircraft climbs through 
1,000 ft. AGL 

Boeing 757, contact Departure Control. 

Crew checks in on Departure 
Control 

Boeing 757, Roger. Say altitude. 

Crew reports altitude 

Boeing 757, Roger. 

Aircraft climbs through 7,000 
ft. MSL 

Boeing 757, contact Seattle Center on 126.1. 

One minute after level at 
8,000 ft. MSL 

Boeing 757, radar contact, climb and maintain 13,000. 

One minute after level at 
13,000 ft. MSL 

Boeing 757, maintain airspeed at 320 knots or greater 
for sequencing. 

Aircraft passes D40/GEG 

Boeing 757, contact Spokane Approach Control on 
124.3. 

Crew contacts Spokane 
Approach Control 

Boeing 757, for traffic ahead, reduce speed to 210, 
then descend and maintain 5,000. Plan an ILS 
Runway 03 approach. Intercept the ILS Runway 03 
localizer via the 10 DME Arc. Report crossing the 216 
radial of the Spokane VOR. 

Crew reports crossing GEG 
radial 

Boeing 757, cleared for an ILS Runway 03 approach 
to the Spokane International Airport. Contact the 
Spokane tower 1 1 8.3 at OLAKE. 

Crew reports OLAKE 

Boeing 757, cleared to land, Runway 03. Wind 340 at 
25 with gusts to 35. RVR Runway 03 2,400. Runway 
03 braking action reported poor by a 737. 

Aircraft slows to 60 knots 

Boeing 757, clear the runway at taxiway C, if able, 
then contact Ground Control on 121.9. 

Crew calls ground control 

Boeing 757, taxi to the terminal via taxiways M and H. 
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ATC Script - Mission Scenario 2 
KGEG/KALW 

Cue I ATC Message 

Crew calls for clearance prior Boeing 757 is cleared to the Walla Walla Airport as 
to taxi filed. Climb and maintain 9,000. Expect 15,000 within 

40 miles of the Spokane VORTAC. Squawk 7572. 
Depa rture Control frequency will be 124.7 

Crew calls ready to taxi Boeing 757, taxi to and hold short of Runway 21 via 

Taxi ways G, M, and D. 

Crew calls ready for takeoff Boeing 757, taxi into position and hold, Runway 21. 

After aircraft is in position on Boeing 757, cleared for takeoff. Fly heading 205 after 

the runway takeoff. 

As the aircraft climbs through Boeing 757, contact Departure Control. 

1.000 ft. AGL 

Crew checks in on Departure Boeing 757, Roger. Say altitude. 

Control 

Crew reports altitude Boeing 757, Roger. 

Aircraft climbs through 7,000 Boeing 757, contact Seattle Center on 126.1. 

ft. MSL 

One minute after level at Boeing 757, radar contact, climb and maintain 15,000. 

9.000 ft. MSL 

One minute after level at Boeing 757, maintain airspeed at 320 knots or greater 

15.000 ft. MSL for sequencing. 

Aircraft passes D40/ALW Boeing 757, contact Tri-Cities Approach Control on 

133.15. 

Crew contacts Tri-Cities Boeing 757, for traffic ahead, reduce speed to 210, 
Approach Control then descend and maintain 5,000. Plan an ILS 

Runway 20 approach. After DATES fly heading 126 to 
intercept the localizer. Report established on the 
localizer inbound. 

Crew reports established on Boeing 757, cleared for an ILS Runway 20 approach to 

the localizer inbound. the Walla Walla Airport. Contact the Walla Walla tower 

118.5 at TRINA. 

Crew reports TRINA Boeing 757, cleared to land, Runway 20. Wind calm. 

RVR Runway 20 2,400. Runway 20 braking action 
reported fair by a 737. 

Aircraft slows to 60 knots Boeing 757, clear the runway at taxiway D, if able, 

th en contact Ground Control on 121.7. 

Boeing 757, taxi to the terminal. 


Crew calls ground control 











ATC Script - Mission Scenario 3 
KBFI/KMWH 


Cue 

ATC Message 

Crew calls for clearance prior 
to taxi 

Boeing 757 is cleared to the Grant County Airport via a 
Kent Two departure, then as filed. Fly runway heading 
after departure. Climb and maintain 2,000. Expect 
17,000 within 6 miles of Boeing Field. Squawk 7573. 
Departure Control frequency will be 1 19.2. Do not 
exceed 250 knots until advised. 

Crew calls ready to taxi 

Boeing 757, taxi to and hold short of Runway 13R at B1. 

Crew calls ready for takeoff 

Boeing 757, taxi into position and hold, Runway 13R. 

After aircraft is in position on 
the runway 

Boeing 757, cleared for takeoff Runway 13R. Wind 210 
at 25 with gusts to 35. 

As the aircraft climbs through 
1,000 ft. AG L 

Boeing 757, contact Departure Control. 

Crew checks in on Departure 
Control 

Boeing 757, Roger. Say altitude. 

Crew reports altitude 

Boeing 757, Roger, radar contact, climb and maintain 
9,000. 

Aircraft climbs through 7,000 
ft. MSL 

Boeing 757, contact Seattle Center on 134.95. 

One minute after level at 
9,000 ft. MSL 

Boeing 757, radar contact, climb and maintain 17,000. 
Cleared to the Grant County Airport via present position 
direct Wenatchee, then a Potholes 1 Arrival. 

Climbing through 15,000 ft. 
MSL 

Boeing 757, maintain airspeed at 320 knots or greater for 
sequencing. 

Aircraft passes EAT 

Boeing 757, contact Seattle Center on 1 26. 1 . 

Crew calls on Seattle Center 
on 126.1 

Boeing 757, slow to 250 knots, then descend and main- 
tain 9,000. 

Aircraft passes D40/MWH 

Boeing 757, contact Grant County Approach Control on 
126.4. 

Crew contacts Grant County 
Approach Control 

Boeing 757, for traffic ahead, reduce speed to 210, then 
descend and maintain 5,000. Plan an ILS Runway 32R 
approach. Report crossing the 182° radial of the MWH 
VORTAC. 

Crew reports crossing the 
1 82° radial of the MWH 
VORTAC 

Boeing 757, descend to 3,000. Maintain 3,000 until 
established inbound on the localizer. Cleared for an ILS 
Runway 32R approach to the Grant County Airport. 
Contact the Grant County Tower on 1 18.1 at PELLY. 

Crew reports PELLY 

Boeing 757, cleared to land, Runway 32R. Wind 200 at 
15. 

Aircraft slows to 60 knots 

Boeing 757, turn left on Runway 21, if able, then contact 
Ground Control on 121.9. 

Crew calls ground control 

Boeing 757, taxi to the terminal via taxiways J, B, and A. 
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ATC Script - Mission Scenario 4 
KBFI/KBFI 


Cue 

ATC Message 

Crew calls for clearance prior 
to taxi 

Boeing 757 is cleared to the Grant County Airport via a 
Kent Two departure, then as filed. Fly runway heading 
after departure. Climb and maintain 2,000. Expect 
17,000 within 6 miles of Boeing Field. Squawk 7574. 
Departure Control frequency will be 1 19.2. Do not 
exceed 250 knots until advised. 

Crew calls ready to taxi 

Boeing 757, taxi to and hold short of Runway 1 3R at B1 . 

Crew calls ready for takeoff 

Boeing 757, taxi into position and hold, Runway 1 3R. 

After aircraft is in position on 
the runway 

Boeing 757, cleared for takeoff Runway 13R. Wind 210 
at 1 5 with gusts to 25. 

As the aircraft climbs through 
1 ,000 ft. AGL 

Boeing 757, contact Departure Control. 

Crew checks in on Departure 
Control 

Boeing 757, Roger. Say altitude. 

Crew reports altitude 

Boeing 757, Roger, radar contact, climb and maintain 
9,000. 

Aircraft climbs through 7,000 
ft. MSL 

Boeing 757, contact Seattle Center on 134.95. 

One minute after level at 
9,000 ft. MSL 

Boeing 757, radar contact, climb and maintain 17,000. 
Cleared to the Grant County Airport via present position 
direct Wenatchee, then a Potholes 1 Arrival. 

Climbing through 15,000 ft. 
MSL 

Boeing 757, maintain airspeed at 320 knots or greater for 
sequencing. 

When crew requests a return 
to Boeing Field 

Boeing 757, turn left to heading 340 and descend to 
12,000. Contact Seattle Approach on 120.3. 

When crew reports in to 
Seattle Approach Control 

Boeing 757, cleared to the Boeing Field Airport via 
present position direct to the PAE VORTAC, then expect 
radar vector to the ILS Runway 13R final approach. 
Descend and maintain 5,000. Reduce speed to 250 
knots. Cross 20 DME Southeast of the PAE VORTAC at 
or above 10,000. 

Aircraft passes PAE 

Boeing 757, for traffic ahead, reduce speed to 210 
knots, turn left to heading 195, descend to 2,200 and 
intercept the localizer. Cleared for the straight-in ILS 
Runway 13R Approach. Contact Boeing Tower at 
NOLLA on 120.6. 

Crew reports NOLLA 

Boeing 757, cleared to land, Runway 13R. Wind 200 at 
15. 

Aircraft slows to 60 knots 

Boeing 757, turn right at B5, if able, then contact Ground 
Control on 121.9. 

Crew calls ground control 

Boeing 757, taxi to the Boeing Company ramp. 
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ATC Script - Mission Scenario 5 
KMWH/KBFI 


Cue 

ATC Message 

Crew calls for clearance prior 
to taxi 

Boeing 757 is cleared to the Boeing Field as filed. Climb 
and maintain 8,000. Expect 16,000 five minutes after 
departure. Leaving 3,000 proceed direct EPH. Squawk 
7575. Departure Control frequency will be 126.4. 

Crew calls ready to taxi 

Boeing 757, taxi to and hold short of Runway 32R at 
Taxiway A. 

Crew calls ready for takeoff 

Boeing 757, taxi into position and hold, Runway 32R. 

After aircraft is in position on 
the runway 

Boeing 757, cleared for takeoff. 

As the aircraft climbs through 
1,000 ft. AGL 

Boeing 757, contact Departure Control. 

Crew checks in on Departure 
Control 

Boeing 757, Roger. Say altitude. 

Crew reports altitude 

Boeing 757, Roger, radar contact. 

Aircraft climbs through 7,000 
ft. MSL 

Boeing 757, contact Seattle Center on 126.1 

One minute after level at 
8,000 ft. MSL 

Boeing 757, radar contact, climb and maintain 16,000, 
cleared direct DUVAL for a DUVAL 1 Arrival to Boeing 
Field. 

One minute after level at 
16,000 ft. MSL 

Boeing 757, maintain airspeed at 320 knots or greater for 
sequencing. 

Aircraft passes GLASR 

Boeing 757, descend at pilot's discretion to 10,000. Cross 
JAKSN at 12,000 and 250 knots. 

Aircraft levels at 10,000 ft 
MSL 

Boeing, contact Seattle Approach Control on 123.9. 

Crew contacts Spokane 
Approach Control 

Boeing 757, for traffic ahead, reduce speed to 210, then 
descend and maintain 5,000. Plan an ILS Runway 13R 
approach to Boeing Field. After JAKSN proceed direct to 
the SEA 325°/20 nm then fly heading 1 60 to intercept the 
localizer. Report established inbound on the localizer. 

Aircraft crosses the SEA 
360° radial 

Boeing 757, Descend and maintain 3,200. 

Aircraft crosses the SEA 
340° radial 

Boeing 757, Descend and maintain 2,200. 

Crew reports inbound on the 
localizer 

Boeing 757, cleared for an ILS Runway 13R approach. 
Contact Boeing Tower 120.6 at NOLLA. 

Crew reports NOLLA 

Boeing 757, cleared to land, Runway 13R. Wind 200 at 15. 

Aircraft slows to 60 knots 

Boeing 757, turn right at B5, if able, then contact Ground 
Control on 121.9. 

Crew calls ground control 

Boeing 757, taxi to the Boeing Company Ramp. 
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APPENDIX B 

Subjects' Background Questionnaires 


This appendix includes both versions of the Pilot Background Question- 
naire filled out by the subjects. One version was completed by Boeing 
training pilots, and the other was completed by United pilots. 



[for Boeing Pilots] 


Snb jeok 


Date: 


PILOT BACKGROUND QUESTIONNAIRE 

We are collecting background data on all pilots who participate in flight 
deck research simulation studies. To help us evaluate the results of this 
experiment, we would appreciate your filling out this questionnaire. Your 
identity will, of course, be completely protected since your name is never 
associated with the data. 


1. How long have you been employed by Boeing? years 

2. How long have you served as a pilot in the following categories? 

Military years General Aviation years 

Airline ears Boeing years 

Other (specify) years 

3. How many 757 hours do you have as: 

Prior to Boeing : 

Captain - Aircraft Simulator 

First Officer - Aircraft Simulator 

Boeing experience : 

As Pilot As Instructor 

Total 757 hours 

Simulator 

Airplane 

Total 767 hours 

Simulator 

Airplane 

Total 747 hours 

Simulator 

Airplane 
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[for Boeing Pilots] 


As Pilot As Instructor 

Total 747-400 hours 

Simulator 

Airplane 

Total 737-100/200 hours 

Simulator 

Airplane 

Total 737-300/400/500 Hrs 

Simulator 

Airplane 

4. How much time have you in other FMC equipped aircraft? 

Aircraft Type 

Total Hours - Simulation Flight 

5. How much time do you spend flying in the following categories? 

Military hours/month 

General Aviation hours/month 

Boeing hours/month 

Other (specify) hours/month 

6. What is your present status? Circle all appropriate answers. 

a. Production Test 

b. Engineering Test 

c. Instructor Pilot 

d. Flight Engineer (full time) 

e. Other (specify) 
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[for Boeing Pilots] 


7. How long have you been in your present status? years 

8. What is your height? ft in. 

9. What is your weight? lbs. 

10. What is your sex? Male Female 

11. What is your age? years 

12. Circle the response which best describes your general health? 

A B C D E 

F air Excellent 

13. How often do you exercise? 


ABODE 
Ox l-2x 3-4x 5-6x 7x 

week week week week week 

14. How often do you exercise at about the same time of day? 

A B C D E 

Rarely Frequently 

15. Briefly describe the type of physical exercise or sports in which you 
participate: 


16. Do you wear corrective lenses? Yes/No 

17. If Yes to #16, what type of lenses do you wear? 

Soft Contacts 

Hard Contacts 

Bifocal Contacts 

Glasses/ 

Bifocals/Verilux 

Other (specify) 
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[for United Pilots] 


Subject: Patel 

PILOT BACKGROUND QUESTIONNAIRE 

We are collecting background data on all pilots who participate in flight 
deck research simulation studies. To help us evaluate the results of this 
experiment, we would appreciate your filling out this questionnaire. Your 
identity will, of course, be completely protected since your name is never 
associated with the data. 

1. How long have you been employed by United? years 

2. How long have you served as a pilot in the following categories? 

Military years General Aviation years 

Airline ears Other (specify) years 

3. How many 757 hours do you have as: 

Captain - Aircraft Simulator 

First Officer - Aircraft Simulator 

4. How many 767 hours do you have as: 

Captain - Aircraft Simulator 

First Officer - Aircraft Simulator 

5. How many 747-400 hours do you have as: 

Captain - Aircraft Simulator 

First Officer - Aircraft Simulator 

6. How much time do you spend flying in the following categories? 

Military hours/month 

General Aviation hours/month 

Boeing hours/month 

Other (specify) hours/month 
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[for United Pilots] 


Page 2 

6. What is your present status? Circle all appropriate answers. 

a. Captain 

b. First Officer 

e. Other (specify) 

7. How long have you been in your present status? years 

8. What is your height? ft in. 

9. What is your weight? lbs. 

10. What is your sex? Male Female 

11. What is your age? years 

12. Circle the response which best describes your general health? 

A B C D E 

Fair Excellent 
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APPENDIX C 

Subjective Questionnaires and Data Summaries 

This appendix includes the subjective data questionnaires given after each 
trial block, with responses summarized on the questionnaires. It also in- 
cludes a copy of the final questionnaire given after all trial blocks were com- 
pleted. Rating and ranking esponses to the trial block and final question- 
naires are summarized at the end with original and transformed scores 
data included. 
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Subject #: 


Date: 


Questionnaire Summary 


Basic EICAS - Display condition l 

This is a mark-the-block questionnaire. For each question, put an X inside 
the block that best describes your opinion. 


Definitions: 
Extremely easy: 
Fairly easy: 

Extremely rapid: 
Fairly rapid: 


Intuitive, no mental effort is required to use. 

Moderate mental workload, some thought is required to 
use. 

Instantaneously. 

With only a moderate delay. 


1. Overall, how easy did you find this display format to use? 



extremely fairly 

easy easy 


Questions 2 and 3 concern use of the ROUND DIAL EICAS indicators. 

2. How easy to use did you find the display elements for monitoring Ni, 
N 2 and EGT? 



extremely fairly 

easy easy 


3. How rapidly were you able to detect an out-of-tolerance condition for 
Ni, N 2 and EGT? ' 


extremely 

rapidly 


fairly 

rapidly 
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Questions 4 and 5 concern use of vertical scale EICAS indicators. 

4. How easy to use did you find the vertical scale display elements for 
monitoring oil, pressure, oil temperature, oil quantity and vibration? 


I I 5 I i I I 

extremely 

easy 


airly 

easy 


How rapidly were you able to detect an out-of-tolerance condition for 
oil, pressure, oil temperature, oil quantity and vibration? 



extremely 

easy 
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Subject #: 


Date: 


Questionnaire Summary 


EICAS Plus Engine Parameter Messages - Display Condition 2 

This is a mark-the-block questionnaire. For each question, put an X inside 
the block that best describes your opinion. 


Definitions: 
Extremely easy: 
Moderately easy: 

Extremely rapid: 
Moderately rapid: 


Intuitive, no mental effort is required to use. 

Moderate mental workload, some thought is required to 
use. 

Instantaneously. 

With only a moderate delay. 


1. Overall, how easy did you find this display format to use? 


extremely 

easy 


moderately 

easy 


Questions 2 and 3 concern use of the ROUND DIAL EICAS indicators. 

2. How easy to use did you find the display elements for monitoring Ni, 
N 2 and EGT? 




2 

4 

3 

1 


extremely 

easy 




moderately 

easy 

3. How rapidly were you able to detect an out-of-tolerance condition for 

Ni, N 2 and EGT? 


1 

2 

2 

4 

1 


extremely 

rapid 




moderately 

rapid 
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Questions 4 and 5 concern use of vertical scale EICAS indicators. 

4. How easy to use did you find the vertical scale display elements for 
monitoring oil, pressure, oil temperature, oil quantity and vibration? 



extremely moderately 

easy easy 


5. How rapidly were you able to detect an out-of-tolerance condition for 
oil, pressure, oil temperature, oil quantity and vibration? 



extremely moderately 

rapid rapid 
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Subject #: 


Date: 


Questionn aire Summary 


Augmented EICAS - Display Condition 3 

This is a mark-the-block questionnaire. For each question, put an X inside 
the block that best describes your opinion. 


Definitions: 
Extremely easy: 
Moderately easy: 

Extremely rapid: 
Moderately rapid: 


Intuitive, no mental effort is required to use. 

Moderate mental workload, some thought is required to 
use. 

Instantaneously. 

With only a moderate delay. 


1. Overall, how easy did you find this display format to use? 


extremely 

easy 


moderately 

easy 


Questions 2 and 3 concern use of the ROUND DIAL EICAS indicators. 

2. How easy to use did you find the display elements for monitoring Ni, 
N 2 and EGT? 


extremely 

easy 


moderately 

easy 


3. How rapidly were you able to detect an out-of-tolerance condition for 
Ni, N 2 and EGT? * 


moderately 

rapid 


extremely 

rapid 
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Questions 4 and 5 concern use of vertical scale EICAS indicators. 

4. How easy to use did you find the vertical scale display elements for 
monitoring oil, pressure, oil temperature, oil quantity and vibration? 

2 

moderately 

easy easy 

5. How rapidly were you able to detect an out-of-tolerance condition for 
oil, pressure, oil temperature, oil quantity and vibration? 


4 



extremely 



extremely 

rapid 


4 

moderately 

rapid 
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Subject #: 


Date: 


Questionnaire Summary 


EICAS Plus monitor and parameter messages - Display 

Condition 4 


This is a mark-the-block questionnaire. For each question, put an X inside 
the block that best describes your opinion. 


Definitions: 
Extremely easy: 
Moderately easy: 

Extremely rapid: 
Moderately rapid: 


Intuitive, no mental effort is required to use. 

Moderate mental workload, some thought is required to 
use. 

Instantaneously. 

With only a moderate delay. 


1. Overall, how easy did you find this display format to use? 


extremely 

easy 


moderately 

easy 


Questions 2 and 3 concern use of the ROUND DIAL EICAS indicators. 


2. How easy to use did you find the display elements for monitoring Ni, 
N 2 and EGT? 




4 

3 

2 

1 


extremely 

easy 




moderately 

easy 

How rapidly were you able to detect an out-of-tolerance condition for 
Ni, N 2 and EGT? 



3 

4 

2 

1 


extremely moderately 

rapid rapid 
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Questions 4 and 5 concern use of vertical scale EICAS indicators. 

4. How easy to use did you find the vertical scale display elements for 
monitoring oil, pressure, oil temperature, oil quantity and vibration? 




1 

2 

3 

4 


extremely 

easy 




moderately 

easy 

5. How rapidly were you able to detect an out-of-tolerance condition for 

oil, pressure, oil temperature, oil quantity and vibration? 



2 

2 

2 

4 


extremely 

rapid 


moderately 

rapid 
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Subject #: 


Date: 


Questionnaire Summary 


Augmented EICAS Plus messages - Display Condition 5 

This is a mark-the-block questionnaire. For each question, put an X inside 
the block that best describes your opinion. 


Definitions: 
Extremely easy: 
Moderately easy: 

Extremely rapid: 
Moderately rapid: 


Intuitive, no mental effort is required to use. 

Moderate mental workload, some thought is required to 
use. 

Instantaneously. 

With only a moderate delay. 


1. Overall, how easy did you find this display format to use? 


extremely 

easy 


moderately 

easy 


Questions 2 and 3 concern use of the ROUND DIAL EICAS indicators. 

2. How easy to use did you find the display elements for monitoring Ni, 
N 2 and EGT? 


extremely 

easy 


moderately 

easy 


How rapidly were you able to detect an out-of-tolerance condition for 
Ni, N 2 and EGT? 


extremely 

rapid 


moderately 

rapid 
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Questions 4 and 5 concern use of vertical scale EICAS indicators. 


4. How easy to use did you find the vertical scale display elements for 
monitoring oil, pressure, oil temperature, oil quantity and vibration? 




2 

3 

3 

2 


extremely 

easy 




moderately 

easy 

5. How rapidly were you able to detect an out-of-tolerance condition for 

oil, pressure, oil temperature, oil quantity and vibration? 



3 

4 

1 

2 


extremely moderately 

rapid rapid 
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Subject #: 


Date: 


Final Questionnaire Summary 

1. Rank the displays according to ease of use, where 1 is easiest to use, 
and 5 is least easy to use: 

4.4 Basic EICAS 

2.2 EICAS plus engine parameter messages (L ENG LOW EGT) 

3.6 EICAS plus monitor messages (MONITOR L EGT) 

2.5 Augmented EICAS (with green NORMAL RANGE bands) 

2.5 Augmented EICAS with geen bands and messages 

2. How useful were the following features added to the standard EICAS 
display? 

2a. Engine parameter messages (L ENG LOW EGT) 




2 


8 

not at all 
useful 




very 

useful 

Monitor messages (MONITOR L EGT) 



1 


5 

4 

not at all 
useful 




very 

useful 

Green bands to show normal range for parameter 


1 

3 

4 

2 


not at all 
useful 




very 

useful 
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3. How easy to interpret were the following features added to the 
standard EICAS display? 

3a. Engine parameter messages (L ENG LOW EGT) 





2 

8 

extremely easy 
to interpret 



very easy 
to interpret 

Monitor messages (MONITOR L EGT) 



1 


5 

4 

extremely easy 
to interpret 



very easy 
to interpret 

Green bands to show normal range for parameter 


1 

2 

1 

5 

1 


extremely easy very easy 

to interpret to interpret 


4. In general, what did you like about the features associated with the 
augmented EICAS display? 


5. In general, what did you dislike about the features associated with 
the augmented EICAS display? 


6. If you have additional comments, please include them here. 
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SUBJECT# 

EICAS + ALERTS AND "MONITOR" MESSAGES 

OVFRAI L EASE OF INTERPRETATION 

1 ROUND-DIAL INDICATORS-EASE OF INTERPRETATION 

ROUND-DIAL INDICATORS-SPEED OF INTERPRETATION | 

VERTICAL-SCALE INDICATORS-EASE OF INTERPRETATION 

VERTICAL-SCALE INDICATORS-SPEED OF INTERPRETATION 


AUGMENTED EICAS + ALERTS AND "MONITOR" MES 

OVERALL EASE OF INTERPRETATION 

ROUND-DIAL INDICATORS-EASE OF INTERPRETATION 

ROUND-DIAL INDICATORS-SPEED OF INTERPRETATION 

VERTICAL-SCALE INDICATORS-EASE OF INTERPRETATION 

VERTICAL-SCALE INDICATORS-SPEED OF INTERPRETATION 
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